- removed comment
Improve sync performance
Issue #924
new
Running "sim sync" to a slow remote filesystem can take a long time due to having to check the time stamp and size of all the remote files. It would be good to speed this up.
One option would be to assume that files are only edited on the local machine, which is a common situation. Each invocation of sim sync to a given machine would record the time at which it completed, and subsequent syncs would only transfer those files which had changed on the local system since then.
A further optimisation would be to avoid statting all the local files by using a filesystem notification API such as fsevents or inotify to determine the required information. An advanced case would also use such a system on the remote side, so you wouldn't have to assume that the remote files were unchanged.
Keyword:
Comments (1)
-
- Log in to comment
To make the "record" idea safer, the target system could also (in addition to what Ian suggests above) record from where it was last synced, and when this happened. Only if both source and target have consistent information should this be used.
Of course, this cannot be the default, since the previous sync may have been aborted due to a transmission error, or the remote system may have deleted files, or the user may have modified files on the remote system.
Installing e.g. DropBox on both systems may be an easy solution.