Commit Graph

264 Commits

Author SHA1 Message Date
Marco Paganini cc4f5ba7ba Add support to toggle bandwidth limits via SIGUSR2.
Sending rclone a SIGUSR2 signal will toggle the limiter between off and
the limit set with the --bwlimit command-line option.
2016-10-30 17:46:59 +00:00
Nick Craig-Wood de73063977 Fix output of crypt objects in logs 2016-10-20 17:46:51 +01:00
Nick Craig-Wood b7875fc02a rclone check: show count of hashes that couldn't be checked #700 2016-10-17 16:48:11 +01:00
Nick Craig-Wood 8a56a6836a Check server time against local time #654 2016-10-08 14:00:50 +01:00
Nick Craig-Wood 83849e0a36 Don't show encrypted password to stop confusion - fixes #656 2016-10-08 11:26:14 +01:00
Nick Craig-Wood d42b38699b Make ResponseHeaderTimeout be `--timeout` not `--contimeout` fixes #766
This was causing a problem with Amazon Drive which often pauses for a
long time after uploads before returning the response.
2016-10-08 10:12:19 +01:00
Nick Craig-Wood d033e92234 Stop single file and `--files-from` operations iterating through the source bucket.
This works by making sure directory listings that use a filter only
iterate the files provided in the filter (if any).

Single file copies now don't iterate the source or destination
buckets.

Note that this could potentially slow down very long `--files-from`
lists - this is easy to fix (with another flag probably) if it causes
anyone a problem.

Fixes #610
Fixes #769
2016-10-07 11:39:39 +01:00
Nick Craig-Wood e94850f322 Fix timeouts not working when set to 0 and firing too often - #766 2016-10-06 10:23:23 +01:00
Nick Craig-Wood aef2ac5c04 Add options for Open and implement Range for all remotes 2016-10-05 21:03:56 +01:00
Nick Craig-Wood 9ea20bac42 Fix accidentally committed test in move code 2016-10-03 20:16:41 +01:00
Nick Craig-Wood 945f49ab5e Make ContentType be preserved for cloud -> cloud copies - fixes #733 2016-10-03 20:02:04 +01:00
Nick Craig-Wood 6c9a258d82 Fix move command
* Delete src files which already existed in dst - fixes #751
  * Fix deletion of src file when dst file older
2016-10-03 19:58:44 +01:00
Nick Craig-Wood f2eeb4301c Make --dump-bodies imply --dump-headers 2016-09-22 08:40:37 +01:00
Nick Craig-Wood 0caac70994 Fix build for < go1.7 2016-09-13 11:36:14 +01:00
Nick Craig-Wood 83ba59749f Make failed uploads not count as "Transferred" - fixes #708 2016-09-12 18:15:58 +01:00
Nick Craig-Wood bbf819e2d1 Move versioncheck so it happens earlier in the compile process. 2016-09-12 17:50:19 +01:00
Nick Craig-Wood 0cb9bb3b54 Redo http Transport code
* Insert User-Agent in Transport - fixes #199
  * Update timeouts to use Context
  * Modernise transport
2016-09-12 17:50:19 +01:00
Nick Craig-Wood 2eaac80c86 b2 with crypt: fix crash when uploading large files - fixes #673 2016-09-05 18:10:01 +01:00
Nick Craig-Wood 3996bbb8cb Version v1.33 2016-08-24 23:02:05 +01:00
Nick Craig-Wood 059743a1b0 crypt: add to integration tests 2016-08-23 17:45:37 +01:00
Nick Craig-Wood a54806e5c1 Fix Move when underlying remote returns ErrorCantMove 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 226c2a0d83 Implement crypt for encrypted remotes - #219 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 9985fc40f4 Make Password parameters obey Optional flag and offer to generate random ones 2016-08-23 17:45:37 +01:00
Nick Craig-Wood b1de4c8cba Implement password Option and re-implement editing
Editing now shows all the options for the fs and asks one at a time
whether they should be changed.
2016-08-23 17:45:37 +01:00
Nick Craig-Wood 6a4e424630 Re-implement Obscure/Reveal so they use AES-CTR encryption 2016-08-23 17:45:37 +01:00
Nick Craig-Wood ebb67c135e Fix listToChan passing nil objects to DeleteFile 2016-08-23 17:45:37 +01:00
Nick Craig-Wood 037a000cc8 b2: fix stats accounting for upload - fixes #602 2016-08-22 21:19:38 +01:00
Nick Craig-Wood 84eb7031bb Implement the rclone cat command 2016-08-18 22:45:32 +01:00
Nick Craig-Wood d7b79b4481 Mark the compiled from source version with -DEV - fixes #627 2016-08-18 21:31:10 +01:00
Nick Craig-Wood b5faaf7116 Fix double close of abort channel - fixes #592 2016-08-18 18:56:57 +01:00
Nick Craig-Wood 0805ec051f Add BasicInfo interface shared between Dir and Object 2016-08-05 17:45:27 +01:00
Nick Craig-Wood ae56df7d4f Add --dedupe-mode only to dedupe command 2016-08-04 21:47:14 +01:00
Nick Craig-Wood 67562081f7 Version v1.32 2016-07-13 17:32:39 +01:00
Nick Craig-Wood 63f6827a0d Version v1.31 2016-07-13 12:28:01 +01:00
Nick Craig-Wood ac3c83f966 Fix integration tests for drive 2016-07-12 21:38:15 +01:00
Nick Craig-Wood b9c8e61d39 Explicitly check the state in tests after writing files
...otherwise Amazon Drive will fail.
2016-07-12 21:36:39 +01:00
Nick Craig-Wood a6056408dd Fix move command - stop it running for overlapping fses - fixes #577
* Make move command check for overlapping remotes and refuse to run
  * Do copy/delete rather than all the copies then all the deletes
  * Doesn't purge the source - this was unexpected behaviour see #512 and #416
  * Add -list-retries flag to test suite to control retries

This changes the semantics of `move` slightly.  However it now errs on
the side of not deleting stuff.
2016-07-12 10:49:37 +01:00
Nick Craig-Wood b9479cf7ab Implement --no-update-modtime flag - fixes #511 2016-07-12 10:46:45 +01:00
Nick Craig-Wood a20d80565b Tidy stats output - fixes #541 2016-07-11 13:04:30 +01:00
Nick Craig-Wood 4bd9932703 Fix wording in verbose copy logs - fixes #574 2016-07-09 10:11:57 +01:00
Nick Craig-Wood 13591c7c00 Redo error handling for sync/copy/move
* Factor sync/copy/move into its own file
  * Make fatal errors abort the sync
  * Make Copy return errors
  * Make Sync/Copy/Move return the last Copy error if there was one
  * Prioritise returning Fatal errors
  * NoRetry errors are returned if no other types of errors
2016-07-04 13:45:53 +01:00
Nick Craig-Wood 28f4061892 Add two more classes of error Fatal and NoRetry
These are for remotes to signal that they have a fatal error and don't
want to continue (eg cap exceeded) or that a particular file shouldn't
be retried for some reason.
2016-07-04 13:45:52 +01:00
Nick Craig-Wood 0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood 9aae143833 Implement cleanup command for emptying trash / removing old versions of files 2016-07-01 16:35:36 +01:00
Nick Craig-Wood 70dc97231e Convert more tests to use assert/require 2016-06-30 15:45:30 +01:00
Nick Craig-Wood af4ef8ad8d Implement --no-traverse flag to stop copy traversing the destination remote.
Refactor sync/copy/move
  * Don't load the src listing unless doing a sync and --delete-before
  * Don't load the dst listing if doing copy/move and --no-traverse is set

`rclone --no-traverse copy src dst` now won't load either of the
listings into memory so will use the minimum amount of memory.

This change will reduce the amount of memory rclone uses dramatically
too as in normal operations (copy without --notraverse or sync) as it
no longer loads the source file listing into memory at all.

Fixes #8
Fixes #544
Fixes #546
2016-06-29 17:38:50 +01:00
Nick Craig-Wood 13797a1fb8 Make retry logs be debug in main copy routine 2016-06-28 08:51:57 +01:00
Nick Craig-Wood 3ad8fb8634 Make DeleteFile and DeleteFiles return errors 2016-06-28 08:51:57 +01:00
Nick Craig-Wood ab43005422 Make NewObject return an error
* make it return an error
  * make a canonical error fs.ErrorNotFound
  * make a test for it
  * remove logs/debugs of error
2016-06-28 08:51:57 +01:00
Nick Craig-Wood b1f131964e Rename NewFsObject to NewObject 2016-06-28 08:51:57 +01:00