Ubuntu / Linux bash shell
do something (echo) with all .txt files
for f in *.txt; do echo ${f}; done;
Example: Search string ‘ABC’ in all .txt files
for f in *.txt ; do echo --${f}-- ; grep "ABC" ${f} ; done
same loop over files, but using a pipe (reading from standard input), and a while-loop
ls *.txt | while read f; do echo ${f}; done;
do something with a small set of files
for f in file1.txt file2.txt file3.txt; do echo ${f}; done;
file1.txt
file2.txt
file3.txt
same, but separately defined list of files
filelist="file1.txt file2.txt file3.txt"
for f in ${filelist}; do echo ${f}; done;
reading list of files from file 'filelist.txt' (for larger number of files)
ls *.csv > filelist.txt # define your list of files
for f in `cat filelist.txt`; do echo ${f}; done;
if a line may include spaces better use a while loop:
cat filelist.txt | while read LINE; do echo "${LINE}"; done
loop over filenames, but remove file extension, see → basename string split
for f in Data/*.txt; do FILENAME=${f%%.*}; echo ${FILENAME}; done;
Data/fileA .txt
Data/fileB .txt
loop over filenames, but remove file extension and path prefix
for f in Data/*.txt ; do FILENAME=`basename ${f%%.*}`; echo ${FILENAME}; done
Data/ fileA .txt
Data/ fileB .txt
Exclude samples that are already processed
process input files Data/*.fastq only if result-files Result/*.txt does not exist
for f in Data/*.fastq; do \
SAMPLE=`basename ${f%%.*}`; \
if [ ! -f Results/${SAMPLE}.txt ]; then \
echo "processing sample ${SAMPLE}"; \
# do something \
fi; \
done
# do something 10 times
for N in {1..10}; do \
echo ${N}; \
done
# double loop over a series of numbers and letters
for N in {1..5}; do
for S in {A..C}; do
echo ${N} ${S};
done;
done;
1 A
1 B
1 C
2 A
2 B
2 C
°°°
general use of: if-then-else (-f check if normal file)
if [ -f /path/to/${SAMPLE} ];
then
# do something if file exist
else
# do something if file does not exist
fi
Copy selected subset of files
# copy all files in filelist.txt to newDir/
ls *.csv > filelist.txt # define your list of files (edit text file)
mkdir newDir # create a new directory
for f in `cat filelist.txt`; do echo copying ${f}; cp ${f} newDir/; done;
# add path if files are not located in working directory
for f in `cat filelist.txt`; do echo copying ${f}; cp path/to/files/${f} newDir/; done;
→ rename multiple files ( add or remove prefix and file extension)
→ recursive loop over all files including files in all sub-directories ( see command → find )