Those nasty little dot files…
When I tried to setup a Gitlab continuous deployment, I encountered a tiny problem: How to remove all files within a Linux directory?
Apparently it is quite easy. Just with a simple
rm -rf dir/*
Wrong!
The above command rm -r dir/* only remove all visible files and directories
from the directory dir. It does not remove hidden files and directories.
Similarly, I also had to copy all files from one directory to another.
Why not delete the whole directory
Someone might say: The whole “deleting all files” process just seems useless! Why not remove the directory as a whole? Something like
rm -rf dir/
mkdir dir
That is true, until you consider dir’s file permissions.
If you remove dir as a whole and creates a new dir,
the old dir’s permissions are not automagically copied to the new one.
Imagine such scenario:
you have a directory dir with owner some_user:some_group and permission 0700,
and you are logged in using root account.
Executing rm -rf dir/ && mkdir dir with root makes a new directory
with owner root:root and permission 0755, quite different from before.
It is not a good idea. And directly copying contents of a directory into another suffers the same issue.
The good thing is, only deleting contents of a directory is possible.
Deleting a directory’s content including hidden ones
A widely used command for deleting directory contents is the
rm -rf dir/{,.}*
It basically expands to
rm -rf dir/* dir/.*
There is still a small issue though:
as bash’s .* matches everything beginning with a dot,
the current directory . and parent directory .. are also matched.
It is not an issue when entering commands manually,
as rm will rage quit those two before continuing on deleting other files.
The problem is, the rm command will exit with error 1.
So we need something different. The below command performs the task perfectly:
rm -rf dir/{..?*,.[!.]*,*}
It expands to
rm -rf dir/..?* dir/.[!.]* dir/*
which includes,
- Everything beginning with two dots except
..itself - Everything beginning with only one dot
- All visible files and directories
Hooray!
Copying contents from a directory to another (Easy trick!)
We can use the same trick {..?*,.[!.]*,*} from above,
but there is an easier alternative:
cp dir/. new_dir/
I have no idea how it works, but it does.
The symbol . does nothing in Bash, and is something handled by cp itself.
My guess is that cp attempts to copy the directory dir/. to new_dir/.
and since . is the name of the current directory,
it copies contents of dir to new_dir directly.
So, a neat little trick than no one knows how it works!
And it is much easier to read than the Bash trick {..?*,.[!.]*,*}.
Other options
You can also modify the behavior of Bash’s * symbol to match hidden files.
shopt -s dotglob
But, ugh, this might cause some issues when you want to
match * with visible files only later.