Want to Sync Directories Into a Synced Directory - Or Something

blahboybaz · February 22, 2025, 4:55am

TLDR;

I WANT TO SYNC SIBLING DIRECTORIES IN MY HOME DIRECTORY INTO THE REMOTE, SYNCED DIRECTORY IN MY HOME DIRECTORY AND HAVE IT ALL SYNC UP TOGETHER NO MATTER WHICH ONE I WORK IN.

Ubuntu Version

~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04.2 LTS
Release:	24.04
Codename:	noble

~$ uname -r
6.8.0-53-generic

Rclone Version

~$ rclone version
rclone v1.69.1
- os/version: ubuntu 24.04 (64 bit)
- os/kernel: 6.8.0-53-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.0
- go/linking: static
- go/tags: none

Rclone Config Mounts Remote File System

~$ rclone config redacted
[MEGA]
type = mega
user = XXX
pass = XXX
### Double check the config for sensitive info before posting publicly

Metadata

Mount Directory Name: MEGA
Mount Directory Path: /home/jake/MEGA
Remote Name: MEGA:
Service File Name: rclone-mega-sync.service
Service File Path: /home/jake/.config/systemd/user/

Please Note: the value user in the service file path /home/jake/.config/systemd/user/ is actually a default directory (I had nothing to do with that).

Goal

To be able to sync sibling directories in the home directory (the mount directory is in my home directory) into the locally mounted remote filesystem (MEGA online storage) so that they can continue to live where they are whilst at the same time being a part of the sync directory and of the cloud storage itself (some directories I don’t want to move into the sync directory to do this).

Goals For Local Storage Mount

Expected Result

To have sibling directories (sibling to ~/MEGA in the home directory) that appear in the ~/MEGA mount directory.

That I am able to work in a sibling directory and have changes reflected in the other two.

To also be able to work in the sibling directory in the ~/MEGA directory or to work in the Web UI and have the changes reflected in the other two.

The entire point of doing all of this was so I could link sibling directories into the mounted sync directory and work in those directories - directly - and locally.

I looked into it a little and it looks like you can use rclone for that too but it may not be the best way to do it (not very performant / drags the system)? I could be wrong but unison SEEMED like it might be real simple to use.

Use unison for Bidirectional Sync?
Use rclone with Cron Jobs or Systemd Timers?
Use lsyncd for Real-Time Sync?

Then again lsyncd is lookin’ kinda sweet too. I think with this one you would sync local directories to the remote and then because the remote is mounted locally that would make that connection? But isn’t that what rclone does?

Can anyone offer some guidance?

wgarcia · February 22, 2025, 5:57am

I’ve been using unison for a similar setup for many years, and I recommend it, never had a problem.

blahboybaz · February 22, 2025, 6:24pm

Thanks. I’ve been thinking a lot about the kind of set up each alternative would lead to (like overall configuration of the system type of thing). Unison is supposed to be bi-directional and deals with conflicts. And I can’t be for sure but the one command example I saw made it look really easy to use. Is it?

wxl · February 22, 2025, 6:33pm

I will also add there’s a beta bi-directional sync in rclone.

Won’t work with the Mega backend, though.

wgarcia · February 22, 2025, 6:37pm

@blahboybaz I think unison is quite easy to use, if you need some help open a new thread if you have any problem with it.

irihapeti · February 23, 2025, 1:56am

I also can recommend unison. I’ve used it for a number of years to sync several devices.

zebra3 · February 24, 2025, 3:46am

If you have a local system being used as the sync server you can set it’s IP as static using Netplan. This approach allows you to avoid the need for dhcp which may not exist on your router. Once your server has a static IP your other systems can depend on it’s ip for syncing.
Just guessing what your setup is!

blahboybaz · February 25, 2025, 12:25am

Kinda sounds complicated (but probably worth it). I think my first thought was - I better not try that or be spending the next week trying to fix all the stuff I broke.

I had a kind of different idea though that SEEMED like it might work?

So I noticed that I can use rclone to mount the entire remote file system. But can I use rclone to mount one specific directory in the remote file system and mount it to where I want?

Because if that is possible then maybe its possible to do both and solve my problem using rclone alone?

At its core this is about keep the directories I want to keep synced on the remote file system. Then whenever work is done in that directory (whether through the web interface or in the local mount) it should update the other location. Has anyone ever tried something like this and could verify if it should work?

PS: I got to wondering something… assuming this even could work would there be an issue where you have to have an rclone config for each mounted directory? In that case I think rclone would be logging in multiple times to the remote (which could cause a problem). Is it possible to do ALL that under a single rclone config - maybe have a big bloated service file or something?

blahboybaz · February 28, 2025, 6:07pm

I really Wish I did.