Rdiff-backup: exlude syntax for lost+found directory

Support and Help
,
1

Have I got the exclude syntax correct for rdiff-backup ?

I am getting a permision denied when performing the backup as it appears to be trying to delete the lost+found directory on the destination drive.

I have tried to exclude the lost+found directory as shown in the code below…

	$rdiff_cmd \
	--exclude-sockets --exclude-device-files --exclude-fifo \
	--exclude ignorecase:'**.ini' \
	--exclude ignorecase:'**.DS_Store' \
	--exclude ignorecase:'**\$RECYCLE.BIN' \
	--exclude ignorecase:'**.AppleDouble' \
	--exclude ignorecase:'**.localized' \
	--exclude ignorecase:'**\lost+found' \
	--include /media/storage/Backup \
        --include /media/storage/Backup/DEL1520_20111229 \
 	--include /media/storage/Backup/External\ 40GB \
 	--include /media/storage/Backup/ProgramInstallation \
        --include /media/storage/Employment \
       --include /media/storage/Media \
	--exclude '**' $source \
	$rdiff_dest
2

More than you wanted but mine is like:

rdiff-backup --exclude ignorecase:'**cache' --exclude ignorecase:'**.trash' --exclude ignorecase:'**lost+found' --exclude ignorecase:'**.thumbnails' --exclude ignorecase:'**.tmp' /home/ /media/backup/home

Hope that helps.
I do run as root though.

3

Thanks 1fallen

Unlike your syntax, I have a backslash ( \ ) before lost+found in mine:

`--exclude ignorecase:'**\lost+found'`

But I still get the error:

PermissionError: [Errno 1] Operation not permitted: b'/media/backStorageON/lost+found'

So the backup does not complete, below is the full response:

NOTE: Starting mirror from source path /media/storage to destination path /media/backStorageON
WARNING: ListError: '/media/backStorageON/lost+found' [Errno 13] Permission denied: b'/media/backStorageON/lost+found'
Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 33, in <module>
    sys.exit(load_entry_point('rdiff-backup==2.2.6', 'console_scripts', 'rdiff-backup')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rdiffbackup/run.py", line 35, in main
    sys.exit(main_run(sys.argv[1:]))
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rdiffbackup/run.py", line 108, in main_run
    ret_val |= conn_act.run()
               ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rdiffbackup/actions/backup.py", line 159, in run
    backup.mirror_compat200(
  File "/usr/lib/python3/dist-packages/rdiff_backup/backup.py", line 39, in mirror_compat200
    DestS.patch(dest_rpath, source_diffiter)
  File "/usr/lib/python3/dist-packages/rdiff_backup/backup.py", line 197, in patch
    ITR(diff.index, diff)
  File "/usr/lib/python3/dist-packages/rdiff_backup/rorpiter.py", line 142, in __call__
    if self._finish_branches(index):
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/rdiff_backup/rorpiter.py", line 179, in _finish_branches
    to_be_finished.end_process_directory()
  File "/usr/lib/python3/dist-packages/rdiff_backup/backup.py", line 641, in end_process_directory
    self.base_rp.rmdir()
  File "/usr/lib/python3/dist-packages/rdiff_backup/rpath.py", line 807, in rmdir
    self.conn.os.chmod(self.path, 0o700)
PermissionError: [Errno 1] Operation not permitted: b'/media/backStorageON/lost+found'

The destination folder is at the root level of the drive…

image

I assume rdiff-backup is trying to delete the lost+found folder on the destination drive ?

Though I’m not sure why the folder is not excluded as I requested ?

Is it ok to have the rdiff-backup destination at the root level of the drive, or does it need to be in a sub folder ?

I have tried it without the backslash before lost+found:

`--exclude ignorecase:'**lost+found'`

But still get the same error !

4

This sounds more like a permissions error at first glance.

5

I would agree with you if I was backing up the lost+found folder, as that requires sudo permissions.

However, I have tried to exclude lost+found so it should not be touching that folder.

So if that folder is excluded, why would it be complaining of permission errors ? It should ignore that folder and therefore there should be no permission errors.

Thats why I questioned my syntax to exclude the lost+found folder, because it doesnt appear to ignore it due to the permission error.

6

Don’t have experience with using rdiff.

But…should this \ not in fact be this / in the lost+found line?

In other words, change from this
--exclude ignorecase:'**\lost+found' \

to this instead
--exclude '**/lost+found' \

Let us know if this works now as you want.

7

Thanks rubi1200

I tried:

--exclude ignorecase:'**/lost+found'

but still have the same error.

From the error I get the impression it is trying to delete the lost+found folder on the destination drive, despite requesting the folder to be excluded:

File “/usr/lib/python3/dist-packages/rdiff_backup/rpath.py”, line 807, in rmdir

[Errno 1] Operation not permitted: b’/media/backStorageON/lost+found’

8

More on permissions Mine as an example:

root@me-lvm:/lost+found# ls -a ..
.                  cdrom  lib64              opt                    sbin                timeshift
..                 dev    lib.usr-is-merged  .package-cache-mutate  sbin.usr-is-merged  tmp
bin                etc    lost+found         proc                   srv                 usr
bin.usr-is-merged  home   media              root                   sys                 var
boot               lib    mnt                run                    tank

And permissions on mine:

# stat /lost+found
  File: /lost+found
  Size: 16384     	Blocks: 32         IO Block: 4096   directory
Device: 252,0	Inode: 11          Links: 2
Access: (0700/drwx------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-12-18 09:25:02.155742179 -0700
Modify: 2024-12-17 11:53:17.000000000 -0700
Change: 2024-12-17 11:53:17.000000000 -0700
 Birth: 2024-12-17 11:53:17.000000000 -0700

The “lost+found” directory is a construct used by fsck when there is damage to the filesystem (not to the hardware device, but to the fs). Files?Folders that would normally be lost because of directory corruption would be linked in that filesystem’s lost+found directory by inode number. Some of these might be lost directories or lost files or even lost devices. Each filesystem should have its own lost+found directory, but you might be looking at a system with only one filesystem. In general, you should hope that the directory is empty; but if there is corruption, be thankful that in many conditions files can be recovered after fsck places them here.

More on that here:
https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/lostfound.html

Permissions as I understand them differ on why or How they are present or created.

1 Like
9

Thanks 1fallen, but if I’ve excluded lost+found then it shouldnt be trying to do anything with it (assuming I’ve used the correct syntax to exclude and I’ve tried different syntax’s with no luck).

I don’t need to or want to use sudo because the files that are being backed up are documents, spreadsheets, PDFs, pictures, movie clips etc

Im trying again, put instead of the destination being at the root (i.e. /media/backStorageON) I have added a sub-folder which doesn’t have the lost+found folder i.e. /media/backStorageON/Storage)

But I still don’t understand why rdiff-backup is trying to modify/delete the lost+found folder, when I’ve excluded it.

As its used by fsck, rdiff-backup should not be doing anything with the lost+found folder anyway, otherwise it will surely corrupt that folder !

10

My guess is it is being made by rdiff, ie: a new file or system.
Did you read through all the pages in my link?

And if snaps or snapd is around I can’t help on that, I’m snap-less. :innocent:

11

About rdiff-backup I can add:

include & exclude is telling rdiff-backup what to read (get) from the source side. Then, the final directive of the command is where to write it to.

I suspect as @1fallen has said, a permission problem is happening.

If you want rdiff-backup to maintain ownership & permissions then you must run it with elevated priviledges. You may be bob trying to back up sally (another user’s file), or root, or, or, and you want those ownerships to stay intact. Bob can’t do this with a good outcome unless bod has root priviledges.

To maintain ownerships, you would need to run with sudo, or use a script owned by root, or an elevated user with sudo rdiff-backup only priviledges (remote login method I use in remote pull/push backup scenarios), or easier yet with system level cron. Running rdiff-backup with sudo, or one of the other elevated methods, does not make root the owner on your target. It will ensure the ownerships/permissions on your target matches your source.

Taken from a resource here: https://rdiff-backup.net/
"
Rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership (if it is running as root), modification times, acls, eas, resource forks, etc.
"

2 Likes
12

I managed to fix the problem by ensuring the rdiff-backup destination folder was empty.

It seems that if the destination folder is not empty, then rdiff-backup will try to remove those folders in order to mirror the source folder (?)

I understand why sudo would be needed when using rdiff-backup.

However, the files I am backing up only have the home user permissions. I proved this by finding the users and groups of all the files in the source folder using…

find -printf "%u\n" | sort | uniq -c
find -printf "%g\n" | sort | uniq -c

The files types are documents, spreadsheets, pdfs, media etc, theres no OS or hard links etc.

These files were originally backed up from Windows and Mac machines to the drive in my Ubuuntu server. And now I’m backing up those files from the drive in my Ubuntu server to an external drive (i.e. a second backup).

This is why I am able to use rdiff-backup without sudo for this particular backup.

I’ve proved it works without sudo (0 errors) but I intially ran into issues when the destination folder was not empty when making the first (and future?) rdiff-backups

It was the frist time I tried with the destination folder at the root of the drive (which has the lost+found folder). I never had a problem when the destincation folder was empty.

Although I was suprised rdiff-backup would try to do anything with the lost+found folder

13

Would you need to escape the plus symbol so that its read as text. Just a suggestion

14

Thanks, how do I do that ? What is the syntax ?

15

Just put a

\

before the plus sign

1 Like
16

I tried the escape symbol but it made no difference

I have been playing around with it and found out whats going on, but not sure why:

  1. If I use sudo the rdiff-backup is sucessfull BUT it does delete the lost+found folder in the root of the destination drive

  2. In fact rdiff-backup deletes any files or folders in the destination folder which are not in the source folder BUT this only happens during the initial rdiff-backup.

  3. I have to use the --force parameter for the intial backup because there is obviously no rdiff-backup-data folder. Without the --force parameter it gives the following error:
    ERROR: Target path '/media/backGeneralON' does not look like a rdiff-backup repository, call with '--force' to overwrite

  4. Once the intial back is complete (i.e. the rdiff-backup-data folder exists), then further rdiff-backups DONT delete any files folders in the destination folder that arent in the source folder. I can recreate the lost+found folder (using sudo mklost+found) or any other files and folders in the destination and rdiff-backup just ignores it.

So this issue I am having only seems to happen with the initial rdiff-backup.

My current solution is to make a sub folder on the destination drive so that its an empty folder.

I dont know whether rdiff-backup is supposed to work like this or if its a bug ?