rsync crash when remote filter rule changes

Issue #549 closed
anonymous created an issue

Hi,

I have setup a nightly backup pull via rsync like this:

$ rsync --version
rsync  version 3.0.7  protocol version 30
$ cd /backup/target
$ rsync  -axvSAHX  --delete --numeric-ids --relative --delete-excluded --bwlimit=10000 --filter='. /backup/conf/filter_glaukos_home'  root@glaukos:/exports/./data/   ./



Today noon I wondered why it's is still not ready.
After finding out that it was caused by a very large tmp dir of a user I told him to fix his local .rsync-filter.

While rsync was still pulling from the guilty "tmp-2011-09-05/" he changed his local filter from
...
+ /download*.xz
- /tmp
+ /tmp*.xz
...
to
...
+ /download*.xz
- /tmp*
+ /tmp*.xz
...

and moreover he deleted the whole directory "tmp-2011-09-05".

And rsync exited this way:

data/data-source/rolf/tmp-2011-09-05/99999_ICGE@wessa_2011-08-16.annot
data/data-source/rolf/tmp-2011-09-05/99999_ICGE@wessa_2011-08-16.rinse
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-28.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-28.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-29.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-29.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-30.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-30.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-31.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-07-31.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-01.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-01.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-04.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-04.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-05.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-05.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-06.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-06.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-07.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-07.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-08.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-08.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-10.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-10.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-11.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-11.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-12.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-12.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-18.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-18.rinse"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-20.annot"
file has vanished: "/exports/data/data-source/rolf/tmp-2011-09-05/999_IUDMIIF@boe_2011-08-20.rinse"
invalid modifier sequence at 't' in filter rule: -/tmp
rsync error: syntax or usage error (code 1) at exclude.c(829) [sender=3.0.7]
rsync: connection unexpectedly closed (130345973884 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) [receiver=3.0.7]
rsync: writefd_unbuffered failed to write 5 bytes to socket [generator]: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(1530) [generator=3.0.7]
----------------------------------------------------------------------------
rsnapshot encountered an error! The program was invoked with these options:
/usr/bin/rsnapshot -c /backup/conf/rsnapshot_glaukos_home_data.conf sync
----------------------------------------------------------------------------
ERROR: /usr/bin/rsync returned 12 while processing root@glaukos:/exports/./data/



This is IMO a very critical issue, since users may crash my backup every day.


cu,
Rudi

Keyword:

Comments (6)

  1. Frank Löffler
    • removed comment
    • changed watchers to anonymous

    This bug was submitted to the bug tracker of the Einstein Toolkit / Cactus Toolkit. Is this in any way relevant to these projects, other than they might use rsync as well? If not I suggest to report this bug in the proper place, most likely http://rsync.samba.org/bugzilla.html.

  2. anonymous reporter
    • removed comment

    Ooooops, wtf, should go to bed. Can't reproduce how I've found this one when looking for rsync. So this bug can clearly be closed from my side ;)

    apologizing, Rudi

  3. anonymous reporter
    • removed comment

    Replying to [comment:4 knarf]:

    Thanks for replying, and good night. :)

    At least I could reproduce my stupidness now: Googling for my rsync error "invalid modifier sequence at 't' in filter rule", first hit showed your #461 and since it's not my bug I've opened a new one ...

    good night anyway!

  4. Log in to comment