Can rsync 'merge' 2 source folders to 1 destination folder with the delete argument?

Support and Help
,
1

I am looking for a way to ‘merge’ two source folders into a single destination folder and delete anything in the destination folder thats not in both source folders.

I expect any solution would need to be written in a bash script, which I have some experience with but not sure how to implement this requirement (assuming its even possible?).

For example, consider the following situation before running rsync…

SOURCE

Myfolder (on HDD X)

  • File1
  • File2
  • File3
  • File4
  • File5

Myfolder (on HDD Y)

  • FileA
  • FileB
  • FileC
  • FileD
  • FileE

DESTINATION

Myfolder (on HDD Z)

  • File7
  • File8
  • File9

After running rsync, I want the destination folder to contain…

Myfolder (on HDD Z)

  • File1
  • File2
  • File3
  • File4
  • File5
  • FileA
  • FileB
  • FileC
  • FileD
  • FileE

i.e.

  • it copies File1 to File5 from Myfolder (on HDD X) to Myfolder (on HDD Z)
  • it copies FileA to FileE from Myfolder (on HDD Y) to Myfolder (on HDD Z)
  • it deletes File7 to File9 from Myfolder (on HDD Z)

If I were to rsync the source Myfolder (on HDD X) to the destination Myfolder (on HDD Z), it would copy File1 to File5 and delete File7 to File9.

But then if I were to rsync the source Myfolder (on HDD Y) to the destination Myfolder (on HDD Z), it would copy FileA to FileE and delete File1 to File5.

2

I reckon that you have to carefully manage the delete on destination flag

Copy files 1 -5 to destination (without deletion)
Copy files A - E to destination (without deletion)

The next run, add the delete on destination flag
Of course, you can easily experiment with the dry-run flag

3

Thanks tea-for-one

I understand the first few steps you suggested…

Step 1. Copy files 1 -5 to destination (without deletion)
Step 2. Copy files A - E to destination (without deletion)

Then step 3 will be…

Step 3. Copy files 1 -5 to destination (with deletion)

But at step 3, it will delete files A-E because they are not in the source folder as this only folder contains files 1-5

4

Ah, I see what you mean
I didn’t write my suggestion clearly enough

Once the destination only contains
Files 1-5
Files A-E
Then, you can rsync Files 1-5 without deletion
Or rsync Files A-E again without deletion

Alternatively, place Files 1 -5 in a destination folder called Numeric
Then, place Files A-E in a destination folder called Alphabetic

Use rsync at folder level

5

Thanks tea-for-one, but rsync with deletion will be required to keep a mirror of the source folders which usually changes (files added and removed).

Placing all the files in a single folder is not suitable because they are large files, which is why they are currently spread over 2 hard drives.

6

I posted on stack exchange where Jim L provided the solution…

https://unix.stackexchange.com/questions/789797/can-rsync-merge-2-source-folders-to-1-destination-folder-with-the-delete-argum

It was as simple as defining two source paths in the rsync command (I didnt realise rsync could do this) !

1 Like
7

Thanks for sharing.

This may prove useful for others in the future.

1 Like