This special remote type rsyncs file contents to somewhere else.

Setup example:

# git annex initremote myrsync type=rsync rsyncurl=rsync://rsync.example.com/myrsync keyid=joey@kitenet.net
# git annex describe myrsync "rsync server"

Or for using rsync over SSH

# git annex initremote myrsync type=rsync rsyncurl=ssh.example.com:/myrsync keyid=joey@kitenet.net
# git annex describe myrsync "rsync server"

configuration

These parameters can be passed to git annex initremote to configure rsync:

  • encryption - One of "none", "hybrid", "shared", or "pubkey". See encryption.

  • keyid - Specifies the gpg key to use for encryption.

  • rsyncurl - Required. This is the url or hostname:/directory to pass to rsync to tell it where to store content.

  • shellescape - Optional. Set to "no" to avoid shell escaping normally done when using rsync over ssh. That escaping is needed with typical setups, but not with some hosting providers that do not expose rsynced filenames to the shell. You'll know you need this option if git annex get from the special remote fails with an error message containing a single quote (') character. If that happens, you can run enableremote setting shellescape=no.

The annex-rsync-options git configuration setting can be used to pass parameters to rsync.

annex-rsync-transport

You can use the annex-rsync-transport git configuration setting to choose whether we run rsync over ssh or rsh. This setting is also used to specify parameters that git annex will pass to ssh/rsh.

ssh is the default transport; if you'd like to run rsync over rsh, modify your .git/config to include

    annex-rsync-transport = rsh

under the appropriate remote.

To pass parameters to ssh/rsh, include the parameters after "rsh" or "ssh". For example, to configure ssh to use the private key at /path/to/private/key, specify

     annex-rsync-transport = ssh -i /path/to/private/key

Note that environment variables aren't expanded here, so for example, you cannot specify -i $HOME/.ssh/private_key.

Hi, I would like to see a example of setting up / using e.g. rsync.net as a repo.

Please also provide a highlevel description of how the rsync repo fits in with git-annex.

Q1. can you adds files to the remote rsync repo, and will they be detected and synced back ? Q2. is all the git history rsync'd to remote ? how do i recover if i loose all data except the remote rsync repo ?

Comment by diepes Fri Mar 29 15:57:20 2013
No, special remotes do not contain a copy of the git repository, and no, git-annex does not notice when files are added to remote rsync repositories. Suggest you read special remotes.
Comment by http://joeyh.name/ Fri Mar 29 17:12:47 2013
  • It sound to me it would be 1st prize if the cloud provider supported the git-annex functionality over ssh.
  • Then it could be a full git-annex repo, and used for recovery if my laptop with the git info gets lost.
  • From special remotes "These can be used just like any normal remote by git-annex"
  • Your comment "No, special remotes do not contain a copy of the git repository"

  • so a special remote is

  • "A. just a remote filesystem, that contains the objects with sha1 names ? "

  • "B. there is no git info and thus no file name & directory details or am i missing something ?"
  • "C. you can't use the remote as a cloud drive to sync changes e.g. file moves, renames between two other git-annex repositories ? (no meta data)"
  • "D. the data on the cloud/rsync storage can not be used directly it has to moved into a git-annex capable storage location. "

  • Would it not be better to mount the remote storage over ssh(sshfs) and then use full git-annex on mounted directory ?

Comment by diepes Fri Mar 29 22:09:49 2013

how would i use rsync.net as a full repository ? (annex files and git repo)

It support's rsync, and I discovered now git as well.

I had a look at the webapp and can't figure out how to set it up, so I can have multiple remote's sync to rsync.net for annex files and git sync.

Link: http://www.rsync.net/resources/howto/github_integration.html

Comment by diepes Mon May 13 14:44:53 2013

rsync has a --daemon mode with a simple challenge-response authentication but no encryption. This offers a nice lightweight alternative to ssh, especially when we store/transfer encrypted content anyways. Is this already supported in git-annex, if yes how to set it up?

Comment by cehteh Sat Jul 27 01:35:37 2013
The first example on this page shows using rsync:// to store files on a system using the rsync daemon.
Comment by http://joeyh.name/ Sun Jul 28 00:11:38 2013

If I get it correctly, it is mainly useable as a backup, which will accumulate contents of the objects managed by git-annex over time.

It would be great to have a use case illustrating its use in concrete matters. Thanks in advance.

Comment by http://olivier.berger.myopenid.com/ Sat Aug 17 17:40:47 2013

There are many use cases for a rsync special remote. You could use it as a backup. You could use it to archive files offline in a drive with encryption enabled so if the drive is stolen your data is not. You could git annex move --to rsyncremote large files when your local drive is getting full, and then git annex move the files back when free space is again available. You could have one repository copy files to a rsync remote, and then git annex get them on another repository, to transfer the files between computers that do not communicate directly. The git-annex assistant makes it easy to set up rsync remotes using this last scenario, which is referred to as a transfer repository, and arranges to drop files from the transfer repository once they have been transferred to all known clients.

None of these use cases are particular to rsync remotes. Most special remotes can all be used in these and other ways. It largely doesn't matter for your use what underlying transport the special remote uses.

Comment by http://joeyh.name/ Thu Aug 22 18:02:00 2013
Thanks @joeyh. I've taken the liberty to add your use case description to special remotes. Hope this helps.
Comment by http://olivier.berger.myopenid.com/ Thu Aug 22 20:23:13 2013
@joey I don't understand you last comment where you state that special remotes can act as transfer repositories "to transfer the files between computers that do not communicate directly". If there's no communication, ie git pushes or pulls, between the computers then they don't know what file names the files on the special remote map to. They need to somehow communicate the git repo too, don't they?
Comment by http://cstork.org/ Sun Aug 25 20:59:41 2013
@Chris yes in that case you still need, a central git repository (which need not be on a host that supports git-annex), or the assistant can use xmpp to sync the git data.
Comment by http://joeyh.name/ Mon Aug 26 18:36:03 2013
Comments on this page are closed.