MirrorSave
/
rclone
mirror of https://github.com/rclone/rclone.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,460 Commits 180 Branches 116 Tags 325 MiB
Commit Graph

353 Commits

Author SHA1 Message Date
Nick Craig-Wood 1ac6dacf0f about: complete other providers and re-work internals 
* Implement about for:
    * local, crypt, cache, drive, swift, hubic, onedrive, pcloud, dropbox
  * Implement `--json` and `---full` flag for `rclone about`
  * change About interface to return a Usage structure
  * Remove operations.About as it is too thin an interface
  * Implement Integration test

Relates to #1138 and #1564
 2018-04-17 21:53:27 +01:00
a-roussos 94e277d759 about: add new command 'about' to get quota info from a remote 
Implemented for drive only.

Relates to #1138 and #1564.
 2018-04-17 21:50:14 +01:00
Nick Craig-Wood 2b7957cc74 vfs: Only make the VFS cache if --vfs-cache-mode > Off 
This stops the cache cleaner running unnecessarily and saves
resources.

This also helps with issue #2227 which was caused by a second mount
deleting objects in the first mounts cache.
 2018-04-16 17:06:41 +01:00
Matthew Holt 9e4cd55477 size: Add --json flag 2018-04-13 13:38:06 +01:00
Nick Craig-Wood dcf8334673 fs: add --dump goroutines and --dump openfiles 
These are developer flags useful for tracking down resource leaks.
 2018-04-11 20:55:58 +01:00
Nick Craig-Wood 06e3fa3aba mounttest: reduce duplicated code and improve test output #2154 
The written out list of tests was replaced with a nested test for
mount and cmount. The tests for each VFS cache mode were also replaced
with nested tests which makes the output and the code much cleaner.
 2018-04-08 15:04:14 +01:00
Nick Craig-Wood be54fd8f70 Remove builds conditional on go1.7 since that is now guaranteed #2154 
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
 2018-04-07 11:42:55 +01:00
Nick Craig-Wood e5be471ce0 Use io.SeekStart/End/Current constants now for go1.7+ #2154 2018-04-07 11:42:36 +01:00
Nick Craig-Wood 80588a5a6b Replace "golang.org/x/net/context" with "context" for go1.7+ #2154 2018-04-07 11:42:08 +01:00
Matt Holt e13f65b953 serve restic: Print actual listener address 2018-04-04 14:56:26 +01:00
Alexander Neumann 5b8977a053 serve restic: Disallow overwriting files in append-only mode - Fixes #2195 
* Disallow overwriting files in append-only mode
* Add tests for append-only mode
 2018-04-04 14:49:13 +01:00
Steve Kriss 2c2bb0f750 cmd/serve/restic: add append-only mode 2018-03-30 19:54:52 +01:00
Stefan a8267d1628
link: allow creating public link to files and folders - closes #1562 2018-03-29 09:10:19 +02:00
Keith Goldfarb 6c10312c75 ncdu: added a "refresh" key - for #2174 
Added Control+L key to refresh screen. Not sure if this is the
best choice, but it appears to be somewhat common.
 2018-03-26 21:02:39 +01:00
Nick Craig-Wood 98a924602f mount, cmount: set --attr-timeout default to 1s - fixes #2157 
This  works around these 3 problems:

  * rclone using too much memory #2157
  * rclone not serving files to samba
    * https://forum.rclone.org/t/rclone-1-39-vs-1-40-mount-issue/5112
  * excessive time listing directories #2095
 2018-03-23 22:42:51 +00:00
Nick Craig-Wood 4f6f07c074 cmount: fix error handling for Open/OpenDir 2018-03-21 19:44:30 +00:00
Nick Craig-Wood 1afac32d80 serve restic: script for running integration test against all remotes 2018-03-18 19:15:39 +00:00
Nick Craig-Wood 26fbd00b4f serve restic: don't buffer the JSON output in memory for the list command 2018-03-18 16:26:58 +00:00
Nick Craig-Wood 1313b529ff serve restic: use ListR (--fast-list) if available 
For Restic's use case, --fast-list will use less transactions and
calling ListR directly means we can avoid the usual memory overhead.
 2018-03-18 16:22:05 +00:00
Nick Craig-Wood 82e835d6fc serve restic: make it easy to run integration tests against any remote 
Just `cd cmd/serve/restic` then `go test -v -remote TestRemote:`
 2018-03-18 14:23:56 +00:00
Nick Craig-Wood fa867a9a4c serve restic: implement accounting for uploads and downloads 
This means the bandwidth stats will be correct and the bandwidth
throttling will work correctly.  This was forgotten as a previous
iteration of the code was using the higher level operations.Rcat which
took care of this.
 2018-03-18 14:19:43 +00:00
Stefan 86892467d9
config: load config file only on first access (closes #1659, closes #2096) (#2147) 2018-03-17 12:36:30 +01:00
Nick Craig-Wood 34c45a7c04 mount, cmount: remove addition of O_CREATE to flags on file open #2141 
Previously this was adding it in to all file opens which was causing
inefficiencies under Windows where it stats the file using
open/fstat/close.

This change will make stat operations run much quicker under Windows
as they won't have to open the underlying file

This problem was introduced in61b6159a05336bd7ba105766de2d2ff171f7fb81
where we added O_CREATE to all file opens and creates.
 2018-03-15 20:48:56 +00:00
Alexander Neumann 84776c4e43 serve/restic: Remove log message on Close 2018-03-14 21:50:33 +00:00
Nick Craig-Wood c1a3e363a6 mount: return ENOSYS rather than EIO on attempted link 
This fixes FileZilla accessing an rclone mount served over sftp.

See: https://forum.rclone.org/t/moving-files-on-rclone-mount-with-filezilla/5029
 2018-03-14 21:10:20 +00:00
Nick Craig-Wood 7ccc6080b0 serve restic: add more info to GET request error 2018-03-14 21:09:47 +00:00
remusb 97b48cf988 rc: add support for Go 1.6 2018-03-14 22:58:20 +02:00
Nick Craig-Wood 86e5a35491 Implement Remote Control for rclone #2111 
This implements a remote control protocol activated with the --rc flag
and a new command `rclone rc` to use that interface.

Still to do
  * docs - need finishing
  * tests
 2018-03-14 22:58:20 +02:00
Nick Craig-Wood 8bb2854fe4 httplib: allow the flags to be prefixed when instantiating a server 2018-03-14 22:58:20 +02:00
Nick Craig-Wood 4e90ad04d5 serve restic: only accept v2 API requests for list 2018-03-11 17:35:01 +00:00
Alexander Neumann fa003e89b6 serve restic: When listing return empty list instead of 'null' 2018-03-11 14:48:46 +00:00
Alexander Neumann 5114b11d6f serve restic: add http2 server on stdin/stdout 2018-03-11 14:48:46 +00:00
Alexander Neumann f832433fa5 serve restic: Return empty list for non-existing dirs 2018-03-11 14:48:43 +00:00
Nick Craig-Wood d073efdc6c serve restic: serves a remote in restic REST API format 2018-03-11 14:43:03 +00:00
Nick Craig-Wood 9e48748182 httplib: Note that authentication is a good idea for non localhost 2018-03-11 14:38:54 +00:00
Nick Craig-Wood 66c69fe620 mount: wait longer for consistency after rm in tests 2018-03-09 23:15:38 +00:00
Nick Craig-Wood 7713acf23d mount: skip failing test TestFileModTimeWithOpenWriters on Windows 2018-03-09 23:15:38 +00:00
Nick Craig-Wood 473a388f6d mount: disable failing test TestWriteFileDoubleClose on OSX 2018-03-09 23:15:37 +00:00
Nick Craig-Wood 09c14af6d1 cmd: Fix go routines at exit message to make it less confusing 2018-03-09 17:15:48 +00:00
Jakub Tasiemski acae10cd6f lsjson: add --encrypted to show encrypted name #1765 2018-03-09 08:44:02 +00:00
Remus Bunduc 70f07fd3ac
fs: add ChangeNotify and backend support for it (#2094) 
* fs: rename DirChangeNotify to ChangeNotify

* cache: switch to ChangeNotify

* ChangeNotify: keep order of notifications
 2018-03-08 22:03:34 +02:00
Nick Craig-Wood fc32fee4ad mount, cmount: add --attr-timeout to control attribute caching in kernel 
This flag allows the attribute caching in the kernel to be controlled.
The default is 0s - no caching - which is recommended for filesystems
which can change outside the control of the kernel.

Previously this was at the default meaning it was 60s for mount and 1s
for cmount.  This showed strange effects when files changed on the
remote not via the kernel.  For instance Caddy would serve corrupted
files for a while when serving from an rclone mount when a file
changed on the remote.
 2018-03-04 11:20:22 +00:00
ishuah ebfeec9fb4 mount: run rclone mount in the background - fixes #723 2018-03-04 14:06:07 +03:00
Nick Craig-Wood 54deb01f00 vfs: Make OpenFile and friends return EINVAL if O_RDONLY and O_TRUNC 
Before this change Open("name", os.O_RDONLY|os.O_TRUNC) would have
truncated the file.  This is what Linux does, but is counterintuitive.
POSIX states this is undefined, so return an error in this case
instead.  This preserves the invariant O_RDONLY => file is not
changed.
 2018-02-26 17:04:27 +00:00
Fabian Möller 00adf40f9f
cryptdecode: use Cipher instead of NewFs (#2087) 
* crypt: extract NewCipher out of NewFs
* cryptdecode: make use of crypt.NewCipher

Fixes #2075
 2018-02-25 12:57:14 +01:00
Nick Craig-Wood 334bf49d30 httplib: add Close() method to shut the server down and use it in tests 2018-02-19 15:45:24 +00:00
Nick Craig-Wood d8f78a7266 serve http/webdav: update docs on SSL 2018-02-19 14:08:17 +00:00
Nick Craig-Wood 358c1fbac9 serve http/webdav: support SSL/TLS 2018-02-16 18:28:10 +00:00
Nick Craig-Wood cc9d7156e4 serve http/webdav: add --user --pass authentication #1802 2018-02-16 18:28:10 +00:00
Nick Craig-Wood 221a8a9c5d serve http/webdav: add --htpasswd option for authentication #1802 2018-02-16 18:28:10 +00:00
Nick Craig-Wood 5530662ccc serve http/webdav: factor common http server creation to httplib 2018-02-16 17:48:20 +00:00
Nick Craig-Wood 0f787e43b0 mount: link the nssm service manager for mount under Windows 2018-02-12 11:29:32 +00:00
Nick Craig-Wood 3a7bb7b2df mount: update docs showing --vfs-cache-mode to work around limitations 2018-02-12 11:29:32 +00:00
Jakub Tasiemski 1018e9bb27 cmd: rewrite touch tests #1934 2018-02-02 13:46:56 +00:00
Fabian Möller 3c7a755631 lsjson: explain the Path value in the docs 2018-01-31 20:06:01 +00:00
Nick Craig-Wood c277a4096c mount: don't set modtime twice #2021 2018-01-29 20:49:13 +00:00
Fabian Möller 1f5e23aedb scripts: make absolute paths consistent 
Change absolute binary paths in scripts to /usr/bin/env or make them
relative.
This allows the scripts to be used on linux distributions
like NixOS, where binaries are not located in /usr/ or /bin/.
 2018-01-26 09:39:05 +00:00
Nick Craig-Wood fa500e6d21 lib/atexit: factor from cmd so it can be used by backend/cache #1946 2018-01-25 10:33:00 +00:00
Jody Frankowski 5e73acd40a Clean up mount.go and vfs/help.go docs 
* Title cleanups
* Typos
* `rclone mount vs rclone sync/copy` update with `File Caching`
 2018-01-20 18:14:20 +00:00
Nick Craig-Wood a80d8a21dc vfs: add flags parameter to Dir.Create 2018-01-19 13:18:40 +00:00
Nick Craig-Wood 517bdc719b vfs: make specialized file Open functions private 2018-01-19 11:46:01 +00:00
Nick Craig-Wood 5ad226ab54 fs: Add dir option to fs.Purge #1891 
Purge optional interface needs fixing too.
 2018-01-19 11:45:50 +00:00
Nick Craig-Wood 97c414f025 config/hash: rename more symbols after factoring into own package 2018-01-18 20:27:52 +00:00
Nick Craig-Wood 71722b5b95 config: factor Obscure and Reveal into its own package 2018-01-18 20:19:55 +00:00
Nick Craig-Wood 821be5ebed ncdu: add link to asciinema demo of it in action 2018-01-18 14:22:43 +00:00
Ernest Borowski 5cce74d630 flags: remove --no-traverse flag because it is obsolete - fixes #1813 
Signed-off-by: Ernest Borowski <er.borowski@gmail.com>
 2018-01-18 11:00:25 +00:00
Nick Craig-Wood 11da2a6c9b Break the fs package up into smaller parts. 
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
 2018-01-15 17:51:14 +00:00
Nick Craig-Wood 60afda007b Move dircache, oauthutil, rest and pacer modules into lib 2018-01-12 17:07:38 +00:00
Nick Craig-Wood b8b620f5c2 Move all backends into backend directory 2018-01-12 17:07:38 +00:00
ishuah 0a7731cf0d cryptdecode: added option to return encrypted file names. Fixes #1923 2018-01-11 19:22:40 +03:00
Nick Craig-Wood 0eba37d8f3 lsf: add --files-only and --dirs-only flags 2018-01-06 18:04:24 +00:00
Nick Craig-Wood c74c3b37da lsf: add option to print hashes 2018-01-06 17:53:37 +00:00
Nick Craig-Wood ed20fa5ee7 ls* commands: update docs and add defaults into options for lsf 2018-01-06 17:00:20 +00:00
Nick Craig-Wood 54a9fdf421 ls2: remove in favour of lsf 2018-01-06 14:41:36 +00:00
Jakub Tasiemski 0d041602cf cmd: new command lsf 2018-01-06 14:39:31 +00:00
Nick Craig-Wood 65618afd8c serve/http: fix serving files with : in - fixes #1939 2018-01-05 17:25:05 +00:00
Nick Craig-Wood 2a01fa9fa0 moveto,copyto: clarify error message if source doesn't exist - fixes #1022 2017-12-15 11:37:31 +00:00
lewapm 9c242edc10 rmdirs: add --leave-root flag - fixes #1874 2017-12-13 10:23:54 +00:00
Nick Craig-Wood 2cf808c825 ncdu: fix crashes on empty directories - fixes #1910 
Up arrow or right arrow in an empty directory would crash ncdu
 2017-12-12 13:54:15 +00:00
remusb b48b537325
cache: plex integration, refactor chunk storage and worker retries (#1899) 2017-12-09 23:54:26 +02:00
Laurence e767c9ac9f Fix typo in dbhashsum description 2017-12-03 10:24:33 +00:00
Nick Craig-Wood 6d3df6f172 cmount: make tests more reliable on Windows 2017-11-28 20:39:24 +00:00
ishuah aab8051f50 move: add --delete-empty-src-dirs flag - fixes #1854 2017-11-28 11:38:19 +03:00
Nick Craig-Wood 1248beb0b2 cachestats: Fix nil pointer if not a cache remote - fixes #1855 
Also don't retry or show stats
 2017-11-24 10:22:23 +00:00
Nick Craig-Wood dc1c679c65 mount: support truncate properly 2017-11-20 19:42:35 +00:00
Nick Craig-Wood af50f31f7d mounttest: wait for Release after every Read to stop using in use files under Windows 2017-11-20 12:46:24 +00:00
Nick Craig-Wood 2b666187a6 cmount: disable tests on windows + race detector 
These either hang or produce incorrect results for reasons I haven't
worked out yet.
 2017-11-20 12:46:24 +00:00
Nick Craig-Wood 00b46a8b96 mounttest: wait for files to disappear from the directory listing 2017-11-20 12:46:24 +00:00
Nick Craig-Wood b21f227bd3 mounttest: fix crash when FUSE not present 2017-11-20 12:46:24 +00:00
Nick Craig-Wood e98e550021 mounttest: wait for all background Close/Release after writing a file 
The filesystem does a certain amount of things asynchronously waiting
for the file to be released after writing it means everything should
be in a consistent state.
 2017-11-20 12:46:23 +00:00
Nick Craig-Wood b4083b4371 vfs: rename Fsync to Sync and implement Sync on Node and Handle 2017-11-20 12:46:23 +00:00
Nick Craig-Wood eb3415db50 cmount: enable more tests for Windows 2017-11-20 12:46:23 +00:00
Nick Craig-Wood 9fbd8a6419 mounttest: fixes for running under Windows 
* don't mount and unmount between cache runs - WinFSP doesn't suport it
  * use OS paths for opening things
 2017-11-20 12:46:23 +00:00
Nick Craig-Wood 61b6159a05 mount, cmount: add O_CREATE to Open calls since fuse doesn't seem to supply it 2017-11-20 12:46:22 +00:00
Nick Craig-Wood c560017934 vfs: add Path method to Node and use it to stop reading nil DirEntry 
All DirEntry calls now have been checked for nil or converted to use Path.
 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 7c3584f4e6 mountlib: wait for mountpoint to disappear under Windows 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 981cfb1bec mounttest: retry directory listings to account for slow updates on Windows 2017-11-20 12:46:22 +00:00
Nick Craig-Wood dec21ccf63 vfs, cmount: make truncate work properly in the presence or otherwise of open files 2017-11-20 12:46:22 +00:00
Nick Craig-Wood 94adf4f43b cmount: translate FUSE open flags into OS flags 
On Windows the fuse.O_* flags do not have the same values as the
os.O_* flags so translate between the two representations.  They are
mostly the same which is why this hasn't caused a problem before.
 2017-11-20 12:46:22 +00:00
Nick Craig-Wood f5f8c0c438 cmount: make Truncate call the correct Handle or Node method 2017-11-20 12:46:22 +00:00