This special remote type stores file contents in directory.

One use case for this would be if you have a removable drive that you want to use it to sneakernet files between systems (possibly with encrypted contents). Just set up both systems to use the drive's mountpoint as a directory remote.

If you just want two copies of your repository with the files "visible" in the tree in both, the directory special remote is not what you want. Instead, you should use a regular git clone of your git-annex repository.

configuration

These parameters can be passed to git annex initremote to configure the remote:

  • encryption - One of "none", "hybrid", "shared", or "pubkey". See encryption.

  • keyid - Specifies the gpg key to use for encryption.

  • chunksize - Avoid storing files larger than the specified size in the directory. For use on directories on mount points that have file size limitations. The default is to never chunk files.
    The value can use specified using any commonly used units. Example: chunksize=100 megabytes
    Note that enabling chunking on an existing remote with non-chunked files is not recommended.

Setup example:

# git annex initremote usbdrive type=directory directory=/media/usbdrive/ encryption=none
# git annex describe usbdrive "usb drive on /media/usbdrive/"

Thanks for this great tool! I was wondering what the differences are between using type=directory, type=rsync, or a bare git repo for directories?

I guess I can't use just a regular repo because my USB drive is formatted as vfat -- which threw me for a loop the first time I heard about git-annex about a year ago, because I followed the walkthrough, and it didn't work as expected and gave up (now I know it was just a case of PEBKAC). It might be worth adding a note about vfat to the "Adding a remote" section of the walkthrough, since the unstated assumption there is that the USB drive is formatted as a filesystem that supports symlinks.

Thanks again, my scientific data management just got a lot more sane!

Comment by Paul Fri Jun 22 18:10:19 2012

The directory and rsync special remotes intentionally use the same layout. So the same directory could be set up as both types of special remotes.

The main reason to use this rather than a bare git repo is that it supports encryption.

Comment by joeyh.name Mon Jun 25 11:29:29 2012
How do I drop a directory remote after initremote? Say I want to start over and enable chunking (not supposed to be enabled on an existing directory), can I simply git remote rm it?
Comment by dzsino Tue Jan 15 18:29:15 2013
The best way to remove a special remote is to first git annex move --from $remote to get all the content out of it, then git annex dead $remote and finally you can git remote rm $remote
Comment by joeyh.name Thu Jan 17 14:22:26 2013

I tried the suggestion on comment 4, but when I add again a remote with the same path, it gets the same repository identifier and is considered dead. Is that expected?

My use case: I use a usb drive to transfer some large files from one git annex to another, then I use the usb drive for something else and the special remote is removed. Later I want to use the same usb drive again, but when I create the repository, it starts in the dead state.

Comment by nicolas-schodet [myopenid.com] Wed Jun 26 13:52:12 2013

@nicolas, I suspect you are using git annex initremote with the same name that you used for the now dead-and-buried remote. That causes it to be reanimated, which is not what you want.

Since git-annex version 4.20130501, git annex initremote is reserved for creating new remotes, not enabling old ones, so it will refuse to do this. That's to avoid exactly this confusion.

Using git annex initremote with a different remote name, and the same directory should work just fine.

Comment by joeyh.name Wed Jun 26 14:30:39 2013
For the use case you're describing, it might be better to define the usb key as a remote set to "manual." Then, you can copy over the things you want with git annex copy --to=usbkey and when you're done drop everything with git annex drop --from=usbkey and never destroy the remote.
Comment by Andrew Wed Jun 26 14:46:40 2013
Thanks for your answers. You're right, the simplest solution for me is to never remove the remote. If my directory is lost, I realized that I can simply create an empty directory.
Comment by nicolas-schodet [myopenid.com] Wed Jun 26 16:05:20 2013

This is great work. I've developed a serious annex-addiction and now I want to use it everywhere! In particular I was hoping to apply it to this use case:

I have large files/directories (approx 5 TB) on an nfs mount to which is a) write-protected (think "read-only medium") and b) used by non-git users. Both reasons prevent me from setting up a git-annex repos there. However, I would like to use git-annex to keep track of the paths and get/drop files from my different computers.

On one of my servers, I set up a git annex repos, hoping to only manage the structure, the locations, and the number of copies. I don't want to have copies of the 5TB files in that repository, as disk space is not unlimited (just for the sake of making them available to my laptop).

I as banking on using a special remote (either directory or rsync) to tell the git-annex repos where the actual data is.

I am not concerned with data loss, as it is backed up in regular time intervals by our sysadmin.

I tried both directory remote and rsync remote but there seems to be a missing piece (I suppose its add). Any ideas?

This is what I did:

I added the directory remote and an rsync remote

  • $ git annex initremote collections type=directory directory=/my/nfs/dir encryption=none
  • $ git annex initremote rsync type=rsync rsyncurl=ssh://myserver/mnt/nfs/dir encryption=none

the copy command fails without complaints $ git annex copy --from collections

I tried adding virtual files to git annex

  • $ git annex add file/inside/dir

but still any kind of get/copy command does not get any new files

It would be awesome if I could use git-annex for this, to keep track of my copies and copies of copies. And then I could also keep track of data on my write-protected DVDs.

Is there any chance?

Thanks a lot!

-- Laura

Comment by dietz Thu Jul 18 16:57:53 2013
comment 2

@Laura the directory special remote requires files to be in a particular directory structure with special names git-annex comes up with. So you can't use it on an existing tree of files like that.

What you can do is use the web special remote, with a file:// url to point to the files wherever they are stored. So for example, git annex addurl file:///media/dvd/file

Comment by joeyh.name Fri Jul 19 09:54:10 2013

Using the web remote is a pretty nice trick!

Thanks, Joey - I would not have guessed that.

-- Laura

Comment by dietz Sat Jul 20 02:06:31 2013
Comments on this page are closed.