MirrorSave
/
rclone
mirror of https://github.com/rclone/rclone.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
717 Commits 180 Branches 116 Tags 325 MiB
Commit Graph

53 Commits

Author SHA1 Message Date
Nick Craig-Wood a6056408dd Fix move command - stop it running for overlapping fses - fixes #577 
* Make move command check for overlapping remotes and refuse to run
  * Do copy/delete rather than all the copies then all the deletes
  * Doesn't purge the source - this was unexpected behaviour see #512 and #416
  * Add -list-retries flag to test suite to control retries

This changes the semantics of `move` slightly.  However it now errs on
the side of not deleting stuff.
 2016-07-12 10:49:37 +01:00
Nick Craig-Wood 9aae143833 Implement cleanup command for emptying trash / removing old versions of files 2016-07-01 16:35:36 +01:00
Nick Craig-Wood af4ef8ad8d Implement --no-traverse flag to stop copy traversing the destination remote. 
Refactor sync/copy/move
  * Don't load the src listing unless doing a sync and --delete-before
  * Don't load the dst listing if doing copy/move and --no-traverse is set

`rclone --no-traverse copy src dst` now won't load either of the
listings into memory so will use the minimum amount of memory.

This change will reduce the amount of memory rclone uses dramatically
too as in normal operations (copy without --notraverse or sync) as it
no longer loads the source file listing into memory at all.

Fixes #8
Fixes #544
Fixes #546
 2016-06-29 17:38:50 +01:00
Nick Craig-Wood ab43005422 Make NewObject return an error 
* make it return an error
  * make a canonical error fs.ErrorNotFound
  * make a test for it
  * remove logs/debugs of error
 2016-06-28 08:51:57 +01:00
Nick Craig-Wood b1f131964e Rename NewFsObject to NewObject 2016-06-28 08:51:57 +01:00
Nick Craig-Wood 1a87b69376 Get rid of LimitedFs - FIXME needs docs on copying single files 
If remote:path points to a file make NewFs return a sentinel error
fs.ErrorIsFile and an Fs which points to the parent.

Use this to remove the LimitedFs and just add this file to the
--files-from list.

This means that server side operations can be used also.

Fixes #518
Fixes #545
 2016-06-28 08:51:43 +01:00
Nick Craig-Wood d3dd672640 Document recursion requirements for Fses 2016-06-18 14:12:47 +01:00
Nick Craig-Wood df1092ef33 Change Fs.Put so that it must cope with existing files 
This should fix duplicate files on drive and 409 errors on
amazonclouddrive however it will slow down the upload slightly as
another roundtrip will be needed.

None of the other Fses needed adjusting.

Fixes #483
 2016-06-13 19:29:10 +01:00
Nick Craig-Wood 4c5b2833b3 Convert to using github.com/pkg/errors everywhere 2016-06-13 17:43:03 +01:00
Nick Craig-Wood 108760e17b Log -v output to stdout by default - fixes #228 2016-06-04 18:49:27 +01:00
Nick Craig-Wood 1d6698a754 Build tweaks - fixes #484 
* disable CGO for static builds everywhere
  * override Version in release build script
  * don't output symbol table in release binaries
 2016-06-03 20:34:19 +01:00
Nick Craig-Wood dbfa7031d2 Factor Lister into own file, write tests and fix 2016-05-07 17:17:43 +01:00
Nick Craig-Wood c2d0e86431 Add more tests for List() and fix resulting problems 2016-05-07 14:50:35 +01:00
Nick Craig-Wood 68ec6a9f5b Add a directory parameter to Fs.List() 2016-05-06 16:52:34 +01:00
Nick Craig-Wood 753b0717be Refactor the List and ListDir interface 
Gives more accurate error propagation, control of depth of recursion
and short circuit recursion where possible.

Most of the the heavy lifting is done in the "fs" package, making file
system implementations a bit simpler.

This commit contains some code originally by Klaus Post.

Fixes #316
 2016-05-06 16:52:34 +01:00
Nick Craig-Wood 414b35ea56 Change the interface of SetModTime to return an error - #348 2016-03-22 15:56:44 +00:00
Nick Craig-Wood e08e35984c Add help to remote chooser in rclone config - fixes #43 2016-02-21 13:40:16 +00:00
klauspost ef06371c93 Create separate interface for object information. 
Take out read-only information about a Fs in a separate struct to limit access.

See discussion at #282.
 2016-02-19 13:31:09 +00:00
Nick Craig-Wood 85d99de26b Fix typo in error strings 2016-02-09 16:15:50 +00:00
klauspost 78c3a5ccfa Add support for multiple hash types. 
Add support for multiple hash types with negotiation of common hash types for comparison.

Manually rebased version of #277 (see discussion there)
 2016-01-11 13:39:33 +01:00
Nick Craig-Wood ac65d8369e Make fs.CheckClose public to stop duplication 2015-12-17 13:30:58 +00:00
Nick Craig-Wood bed01a303f Add UnWrapper interface and implement in LimitedFs 2015-11-14 08:08:51 +00:00
Nick Craig-Wood 2c2cb84ca7 Make it so optional interface Purge can fail so it can be wrapped 2015-11-14 08:08:51 +00:00
Nick Craig-Wood ff16e0f6df Factor common error handling into fs module 2015-11-06 15:26:58 +00:00
Nick Craig-Wood 4e952af614 Allow spaces in remotes and check remote names for validity at creation time - fixes #171 2015-10-12 17:54:09 +01:00
Nick Craig-Wood e9c915e6fe Fix golint warnings 2015-09-22 18:47:16 +01:00
Nick Craig-Wood a288c2b3a3 Make a retry error wrapper for a plain error 2015-09-09 23:22:41 +01:00
Nick Craig-Wood 92745aa950 Add Root() to Fs interface 2015-09-01 21:49:13 +01:00
Nick Craig-Wood 59ba8f28c8 Implement move command - fixes #35 
* Define Mover interface to move a single object
  * Define DirMover interface to move a directory
  * Implement DirMove operation
  * Add `rclone move` command
  * Tests for Dir Move

To Do
  * Implement Move, DirMover in local, drive, dropbox
  * unit test for Mover
  * unit test for DirMover
 2015-08-28 08:49:16 +01:00
Nick Craig-Wood a96b522958 Implement server side copies if possible - fixes #99 
Add optional fs.Copier interface

Implemented for
  * swift
  * s3
  * drive
  * dropbox
  * google cloud storage
 2015-08-23 21:18:38 +01:00
Nick Craig-Wood fedf81c2b7 Add Name() to Fs interface to return name as passed to NewFs 2015-08-23 13:36:38 +01:00
Nick Craig-Wood 1d9e76bb0f dropbox: remove datastore - Fixes #55 #84 
This means that dropbox no longer stores MD5SUMs and modified times.

Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.

This opens the door for other FSs which don't support metadata easily.
 2015-08-17 17:26:36 +01:00
Leonid Shalupov 3fcff32524 do not print stats in quiet mode - fixes #70 
...unless had some errors or stats interval requested.

Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
 2015-08-10 11:17:54 +01:00
Nick Craig-Wood 90744301d3 Fix package docs so they appear in godoc correctly 2015-06-06 14:24:30 +01:00
Nick Craig-Wood 873db29391 Log all objects more informatively 2015-03-15 15:27:55 +00:00
Nick Craig-Wood b4a0941d4c In remote paths, change native directory separators to / - fixes #37 2015-03-02 17:04:34 +00:00
Nick Craig-Wood 20ad96f3cd windows: Stop drive letters (eg C:) getting mixed up with remotes (eg drive:) 
This was done by stopping the user configuring single letter remotes
and making sure we don't treat single letter remotes as a remote name,
but as a drive letter.
 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 5fb6f94579 drive: fix rate limit exceeded errors - fixes #20 
This is done by pacing the requests to drive and backing them off
using an exponential increase.  Put and Modify operations can now be
retried also.
 2015-02-07 22:32:51 +00:00
Nick Craig-Wood 3e27ff1b95 Add Root List test and fs.Limited tests for single files 2014-07-31 21:35:29 +01:00
Nick Craig-Wood ff91698fb5 Skip tests if test remote not configured 2014-07-31 08:51:39 +01:00
Nick Craig-Wood 2f9f9afac2 fs: Document that Purger returns error on empty directory, test and fix 2014-07-29 17:18:22 +01:00
Nick Craig-Wood a5b0d88608 Make tests for each Fs 
Factor tests out of rclonetest
 2014-07-24 23:13:32 +01:00
Nick Craig-Wood c1a245d1c8 Factor UserAgent to fs and move Version to fs 2014-07-13 19:19:58 +01:00
Nick Craig-Wood 99695d57ab Implement single file operations for all file systems 2014-05-05 22:17:57 +01:00
Nick Craig-Wood 02afcb00e9 Factor Fs.Put into Object.Update and call Update rather than Put if possible 2014-04-18 17:49:01 +01:00
Nick Craig-Wood 92ec29fe3f Factor the generic code into fs and add some more intefaces 2014-03-28 22:26:42 +00:00
Nick Craig-Wood b0064e6405 Change remote format one last time 2014-03-27 17:31:57 +00:00
Nick Craig-Wood 3eda36f6da Factor rclone main() in preparation for tests 2014-03-27 17:26:24 +00:00
Nick Craig-Wood 1b3a49929b Add ability for FS to have a Config helper function run 2014-03-16 13:54:43 +00:00
Nick Craig-Wood 0a108832e2 Remove filesystem flags and put in config file with editor 2014-03-15 16:06:11 +00:00