e7d04fc103
Create fs.Directory interface and use it everywhere
2017-06-30 14:26:59 +01:00
8a6a8b9623
Change List interface and add ListR optional interface
...
This simplifies the implementation of remotes. The only required
interface is now `List` which is a simple one level directory list.
Optionally remotes may implement `ListR` if they have an efficient way
of doing a recursive list.
2017-06-14 16:49:40 +01:00
20da3e6352
Add options to Put, PutUnchecked and Update, add HashOption and speed up local
...
* Add options to Put, PutUnchecked and Update for all Fses
* Use these to create HashOption
* Implement this in local
* Pass the option in fs.Copy
This has the effect that we only calculate hashes we need to in the
local Fs which speeds up transfers significantly.
2017-05-29 12:04:52 +01:00
525220b14e
Add --local-no-unicode-normalization flag
...
Fixes #1411
2017-05-24 07:36:06 +01:00
5355881332
local: fix unnormalised unicode causing problems reading directories #1212
2017-03-16 22:37:56 +00:00
fdaac6df67
local: open files in write only mode so they can write to an rclone mount
...
Fixes #1195
2017-03-02 22:03:07 +00:00
5fba913207
local: fix detection of directories in new object creation
...
This stops the local listing erroring on all symlinks
2017-02-27 17:03:31 +00:00
79e3c67bbd
local, yandex, dropbox: fix NewObject suceeding on a directory #1079
...
Add tests to make it consistent across all remotes
2017-02-25 11:09:57 +00:00
f3c5745468
Add srcRemote and dstRemote parameters to DirMove #954
2017-02-16 17:42:37 +00:00
370f242fa2
local: Fix interaction between -x flag and --max-depth - fixes #1126
...
This was causing the by directory sync to ignore the -x flag because
it was putting directories into the listing which should have been
excluded.
2017-02-13 09:24:29 +00:00
006227baed
Replace -v with -vv where necessary or change Debugf to Logf
2017-02-11 20:27:46 +00:00
80f53176d9
Rename log functions and factor into own file
2017-02-11 17:54:50 +00:00
94947f2523
Implement -L, --copy-links flag to allow rclone to follow symlinks
...
Fixes #40
2017-01-29 13:43:20 +00:00
1fa258c2b4
Define a new Features() method for Fs
...
Optional interfaces are becoming more important in rclone,
--track-renames and --backup-dir both rely on them.
Up to this point rclone has used interface upgrades to define optional
behaviour on Fs objects. However when one Fs object wraps another it
is very difficult for this scheme to work accurately. rclone has
relied on specific error messages being returned when the interface
isn't supported - this is unsatisfactory because it means you have to
call the interface to see whether it is supported.
This change enables accurate detection of optional interfaces by use
of a Features struct as returned by an obligatory Fs.Features()
method. The Features struct contains flags and function pointers
which can be tested against nil to see whether they can be used.
As a result crypt and hubic can accurately reflect the capabilities of
the underlying Fs they are wrapping.
2017-01-16 17:33:25 +00:00
f73ee5eade
Make all config file variables be settable in the environment
...
These are set in the form RCLONE_CONFIG_remote_option where remote is
the uppercased remote name and option is the uppercased config file
option name. Note that RCLONE_CONFIG_remote_TYPE must be set if
defining a new remote.
Fixes #616
2017-01-03 22:42:47 +00:00
e79a5de7df
local: fix Mkdir/Rmdir with a dir on Windows
2016-12-05 18:09:45 +00:00
aaa1370a36
Add directory parameter to Rmdir and Mkdir #100 #831
...
This will enable rclone to manage directories properly in the future.
2016-11-26 12:02:53 +00:00
441951a93b
Stop removing failed upload to cloud storage remotes - fixes #559
...
We do remove a partially written file on local so we don't have
corrupted files lying around.
2016-11-04 21:34:25 +00:00
b35123ba48
Make -x/--one-file-system compile under Windows and add docs
2016-11-03 11:53:49 +00:00
15c9fed60f
local: Implement -x/--one-file-system to stay on a single file system
2016-11-03 11:52:40 +00:00
368cce93ff
Ignore files with control characters in the names - fixes #689
2016-10-18 15:24:29 +01:00
aef2ac5c04
Add options for Open and implement Range for all remotes
2016-10-05 21:03:56 +01:00
3e43ff7414
local: windows - ignore the symlink bit on files
...
This allows files with reparse points to be backed up.
Fixes #614
2016-09-19 17:29:22 +01:00
bb21cf6f0e
local: ignore directory based junction points on windows
...
These are a kind of symlink and rclone doesn't follow symlinks.
Fixes #692
2016-09-19 17:29:22 +01:00
6089f443b9
Fix windows build - fixes #628
...
Try to make clearer the distinction between OS paths and rclone paths
(remotes) so it is harder to muddle them up.
2016-08-20 12:29:54 +01:00
9eeed25418
local: fix filenames with invalid UTF-8 not being uploaded #568
2016-07-15 14:18:09 +01:00
ab43005422
Make NewObject return an error
...
* make it return an error
* make a canonical error fs.ErrorNotFound
* make a test for it
* remove logs/debugs of error
2016-06-28 08:51:57 +01:00
b1f131964e
Rename NewFsObject to NewObject
2016-06-28 08:51:57 +01:00
1a87b69376
Get rid of LimitedFs - FIXME needs docs on copying single files
...
If remote:path points to a file make NewFs return a sentinel error
fs.ErrorIsFile and an Fs which points to the parent.
Use this to remove the LimitedFs and just add this file to the
--files-from list.
This means that server side operations can be used also.
Fixes #518
Fixes #545
2016-06-28 08:51:43 +01:00
e0aa4bb492
Fix incomplete local hashes.
...
Fixes #533
2016-06-19 16:51:49 +02:00
4c5b2833b3
Convert to using github.com/pkg/errors everywhere
2016-06-13 17:43:03 +01:00
c6a79ff72d
Fix remaining places in listing where we were logging errors not returning them
2016-05-30 19:51:15 +01:00
b6f1391da3
Fix new style directory listing on windows
2016-05-30 19:44:15 +01:00
58befe280c
Fix directory name normalisation on OS X
2016-05-28 20:23:37 +01:00
c2d0e86431
Add more tests for List() and fix resulting problems
2016-05-07 14:50:35 +01:00
68ec6a9f5b
Add a directory parameter to Fs.List()
2016-05-06 16:52:34 +01:00
753b0717be
Refactor the List and ListDir interface
...
Gives more accurate error propagation, control of depth of recursion
and short circuit recursion where possible.
Most of the the heavy lifting is done in the "fs" package, making file
system implementations a bit simpler.
This commit contains some code originally by Klaus Post.
Fixes #316
2016-05-06 16:52:34 +01:00
d205dc23e9
Fix oddities using a file in the root - fixes #471
...
* Check return from NewFsObject which caused nil ptr deref
* Correct root directory from "" to string(os.PathSeparator) in getDirFile
2016-05-06 13:52:50 +01:00
7c01bbddf8
Normalise path names for OSX local filesystem
...
Fixes #194 Fixes #451 Fixes #463
2016-05-01 13:13:20 +01:00
414b35ea56
Change the interface of SetModTime to return an error - #348
2016-03-22 15:56:44 +00:00
e57c4406f3
Add mutex to "warned" map.
...
Fixes #385
2016-03-10 15:51:56 +01:00
88cca8a6eb
Simplify literals (after running gofmt -s over the code)
2016-02-29 16:57:23 +00:00
e08e35984c
Add help to remote chooser in rclone config - fixes #43
2016-02-21 13:40:16 +00:00
ef06371c93
Create separate interface for object information.
...
Take out read-only information about a Fs in a separate struct to limit access.
See discussion at #282 .
2016-02-19 13:31:09 +00:00
d0678bc3e5
local: report error on stat in Hash in case file disappeared
2016-01-17 10:46:19 +00:00
78c3a5ccfa
Add support for multiple hash types.
...
Add support for multiple hash types with negotiation of common hash types for comparison.
Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
3c31d711b3
Add local file system option to disable UNC on Windows.
...
This will add an option to disable UNC conversion on Windows to deal with buggy file system implementations like EncFS.
Fixes #261
2016-01-05 17:08:11 +00:00
b257de4aba
Be more constistent with naming in remotes
...
* External objects are called Fs and Object
* Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
16c9fba5de
Fix tests failing on Windows.
...
* ":" is kept when part of a drive.
* Create tests.
* Fix test framework.
2015-10-19 17:36:15 +01:00
ab9f521cbd
Allow '&' and disallow ':' in Windows filenames.
...
Fixes #161
2015-10-05 11:04:25 +02:00
Nick Craig-Wood
Bob Potter
Durval Menezes
Klaus Post
Nick Craig-Wood