Commit Graph

1435 Commits

Author SHA1 Message Date
Nick Craig-Wood cd1735bb10 cache: mark as not supporting metadata 2022-06-29 17:30:37 +01:00
Nick Craig-Wood 866c873daa backend: allow wrapping backend tests to run in make quicktest 2022-06-29 17:30:37 +01:00
Nick Craig-Wood c556e98f49 local: add Metadata support #111 2022-06-29 14:29:36 +01:00
Nick Craig-Wood 22abd785eb s3: implement reading and writing of metadata #111 2022-06-29 14:29:36 +01:00
Nick Craig-Wood a692bd2cd4 s3: change metadata storage to normal map with lowercase keys 2022-06-29 14:29:36 +01:00
Nick Craig-Wood 461d041c4d fstest: remove spurious contents return from PutTestContents and friends 2022-06-29 11:18:02 +01:00
vyloy 326c43ab3f s3: add IDrive e2 to provider list 2022-06-28 09:12:36 +01:00
albertony fdd2f8e6d2 Error strings should not be capitalized
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-23 23:26:02 +02:00
Abhiraj 027746ef6e
drive: moved rclone_folder_id to advanced section - fixes #3463 2022-06-23 22:08:09 +02:00
albertony dbf1234edf docs: skip "Connection" suffix from FTP, SSH/SFTP and HTTP backend names 2022-06-21 23:43:00 +02:00
albertony bc70a95fca docs: consistent capitalization of WebDAV, DLNA, HTTP 2022-06-21 23:43:00 +02:00
Nick Craig-Wood e95dff2fa1 drive: add backend commands exportformats and importformats for debugging 2022-06-21 14:28:53 +01:00
buengese 21c746a56c fichier: parse api error codes and them accordingly 2022-06-19 15:07:33 +02:00
Nick Craig-Wood f7c36ce0f9 s3: unwrap SDK errors to reveal underlying errors on upload
The SDK doesn't wrap errors in a Go standard way so they can't be
unwrapped and tested for - eg fatal error.

The code looks for a Serialization or RequestError and returns the
unwrapped underlying error if possible.

This fixes the fs/operations integration tests checking for fatal
errors being returned.
2022-06-17 16:52:30 +01:00
Scott Grimes 295006f662 opendrive: resolve lag and truncate bugs fixes #5936
Co-authored-by: buengese <buengese@protonmail.com>
2022-06-17 16:48:03 +01:00
Nick Craig-Wood c85fbebce6 s3: simplify PutObject code to use the Request.SetStreamingBody method
In this commit

e5974ac4b0 s3: use PutObject from the aws SDK to upload single part objects

rclone was made to upload objects to s3 using PUT requests rather than
using signed uploads.

However this change missed the fact that there is a supported way to
do this in the SDK using the SetStreamingBody method on the Request.

This therefore reverts a lot of the previous commit to do with making
an unsigned connection and other complication and uses the SDK
facility.
2022-06-16 23:26:19 +01:00
Nick Craig-Wood 5697dbc80f local: fix parsing of --local-nounc flag 2022-06-16 22:13:50 +01:00
Nick Craig-Wood 5e4caa69ce local: make Hash function cancelable via context 2022-06-16 22:13:50 +01:00
Nick Craig-Wood fa48b880c2 s3: retry RequestTimeout errors
See: https://forum.rclone.org/t/s3-failed-upload-large-files-bad-request-400/27695
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 60d87185e1 sftp: add --sftp-set-env option to set environment variables
Fixes #6094
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 78120d40d9 sftp: add --sftp-concurrency to improve high latency transfers
See: https://forum.rclone.org/t/increasing-sftp-transfer-speed/29928
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 95e0934755 sftp: add --sftp-chunk-size to control packets sizes for high latency links
See: https://forum.rclone.org/t/increasing-sftp-transfer-speed/29928
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 1651429041 union: add min_free_space option for lfs/eplfs policies - fixes #6071 2022-06-16 22:13:50 +01:00
Nick Craig-Wood 29e37749b3 union: fix eplus policy to select correct entry for existing files #6071 2022-06-16 22:13:50 +01:00
Nick Craig-Wood 1e1af46a12 union: fix get free space for remotes which don't support it #6071
Before this fix GetFreeSpace returned math.MaxInt64 for remotes which
don't support reading free space, however this is used in various
comparison routines as a too large value, meaning that remotes of size
math.MaxInt64 were never being selected.

This fixes GetFreeSpace to return math.MaxInt64 - 1 so then can be selected.

It also fixes GetUsedSpace the same way however as the default for not
supported was 0 this was very unlikely to have ever caused a problem.
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 1d2fe0d856 union: enable passing of options to upstreams and policies #6071
This factors out the options into a sub package so they can be passed
to upstreams and used in policies.
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 4d72abf389 dropbox: fix nil pointer exception on dropbox impersonate user not found
Fixes #6139
2022-06-16 22:13:50 +01:00
Nick Craig-Wood 411013dbdc drive: add --drive-resource-key for accessing link-shared files 2022-06-16 22:13:50 +01:00
Nick Craig-Wood e87e331f4c drive: make --drive-shared-with-me work with shared drives
Fixes #6247
2022-06-16 22:13:50 +01:00
Maciej Radzikowski 2e91287b2e
docs/s3: add note about chunk size decreasing progress accuracy 2022-06-16 22:29:36 +02:00
buengese 32f913ffbd pcloud: fix cleanup - fixes #3853 2022-06-16 15:45:42 +02:00
Sven Gerber 50c2e37aac
onedrive: add access scopes option
By default, rclone always requests read and write permissions. No matter what settings you configure in the AAD application. This option allows to explicitly request readonly permissions

Migrated read only option to access scope option and set disable_site_permission option to hidden.
2022-06-14 10:21:23 +01:00
m00594701 02b4638a22 backend: add Huawei OBS to s3 provider list 2022-06-14 09:21:01 +01:00
albertony ec117593f1 Fix lint issues reported by staticcheck
Used staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
2022-06-13 21:13:50 +02:00
buengese 74bd7f3381 pcloud: fix about with no free space left 2022-06-13 20:40:15 +02:00
albertony 5006ede266 mailru: use variable type int instead of time.Duration for keeping number of seconds
Fixes problem reported by staticcheck lint tool (v0.3.2):
Poorly chosen name for variable of type time.Duration (ST1011).
2022-06-13 17:42:24 +01:00
buengese 3a20929db4 uptobox: fix root path handling - fixes #5903 2022-06-12 22:36:46 +02:00
buengese cee79f27ee zoho: add Japan and China regions 2022-06-12 15:37:30 +02:00
albertony 5db9a2f831 sftp: use vendor-specific VFS statistics extension for about if available
See #5763
2022-06-08 21:14:53 +02:00
albertony b4091f282a sftp: add support for about and hashsum on windows server
Windows shells like cmd and powershell needs to use different quoting/escaping
of strings and paths than the unix shell, and also absolute paths must be fixed
by removing leading slash that the POSIX formatted paths have
(e.g. /C:/Users does not work in shell, it must be converted to C:/Users).

Tries to autodetect shell type (cmd, powershell, unix) on first use.

Implemented default builtin powershell functions for hashsum and about when remote
shell is powershell.

See #5763

Fixes #5758
2022-06-08 21:14:53 +02:00
Nick Craig-Wood bb6edb3c39 build: update dependencies
Also:

- azureblob: fix compile after API change in upstream library
2022-06-08 18:29:42 +01:00
Noah Hsu ef089dd867
webdav: add SharePoint in other specific regions support 2022-06-08 17:24:35 +01:00
albertony e3d44612c1 jottacloud: error strings should not be capitalized 2022-06-08 17:56:37 +02:00
albertony b2388f1294 jottacloud: refactor endpoint paths 2022-06-08 17:56:37 +02:00
albertony a571c1fb46 jottacloud: refactor naming of different api urls 2022-06-08 17:56:37 +02:00
albertony 01340acad2 jottacloud: add support for upload to custom device and mountpoint
See #5926
2022-06-08 17:56:37 +02:00
albertony 700ca23a71 config: add utility function for backend config with list and custom input 2022-06-08 17:56:37 +02:00
Nick Craig-Wood 4b358ff43b combine: backend to combine multiple remotes in one directory tree
Fixes #5600
2022-06-08 14:57:25 +01:00
Nick Craig-Wood e58d75e4d7 drive: make backend config -o config add a combined AllDrives remote
This adjusts

    rclone backend drives -o config drive:

So that it also emits a config section called `AllDrives` which uses
the combine backend to make a backend which combines all the shared
drives into one.

It also makes sure that all the shared drive names are valid rclone
config names, deduplicating if necessary.

Fixes #4506
2022-06-08 14:57:25 +01:00
Nick Craig-Wood 26db80c270 ftp: revert to upstream github.com/jlaffaye/ftp from our fork
...now all of our patches have been merged #5810
2022-06-08 11:58:32 +01:00