MirrorSave
/
rclone
mirror of https://github.com/rclone/rclone.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,935 Commits 183 Branches 116 Tags 366 MiB
Commit Graph

988 Commits

Author SHA1 Message Date
Nick Craig-Wood ce9028bb5b chunker: set Features.ReadMimeType=false as Object.MimeType not supported 2020-11-29 17:22:43 +00:00
buengese 52688a63c6 jottacloud: don't erroniously report support for writing mime types - fixes #4817 2020-11-29 18:11:43 +01:00
Nick Craig-Wood bcbe393af3 sftp: implement Shutdown method 2020-11-27 17:35:01 +00:00
Nick Craig-Wood 47aada16a0 fs: add Shutdown optional method for backends 2020-11-27 17:35:01 +00:00
Nick Craig-Wood dfadd98969 azureblob,memory,pcloud: fix setting of mime types 
Before this change the backend was reading the mime type of the
destination object instead of the source object when uploading.

This changes fixes the problem and introduces an integration test for
it.

See: https://forum.rclone.org/t/is-there-a-way-to-get-rclone-copy-to-preserve-metadata/20682/2
 2020-11-27 14:40:05 +00:00
Nick Craig-Wood 9d574c0d63 fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
Nick Craig-Wood 2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
Nick Craig-Wood 979bb07c86 filefabric: Implement the Enterprise File Fabric backend 
Missing features
- M-Stream support
- Oauth-like flow (soon being changed to oauth)
 2020-11-25 21:11:29 +00:00
Nick Craig-Wood dfeae0e70a Revert "sharefile: fix backend due to API swapping integers for strings" 
The API seems to have reverted to what it was before

This reverts commit 095c7bd801.
 2020-11-25 20:52:57 +00:00
Nick Craig-Wood f43a9ac17e pcloud: only use SHA1 hashes in EU region 
Apparently only SHA1 hashes are supported in the EU region for
pcloud. This has been confirmed by pCloud support. The EU regions also
support SHA256 hashes which we don't support yet.

https://forum.rclone.org/t/pcloud-to-local-no-hashes-in-common/19440
 2020-11-25 20:46:38 +00:00
Nick Craig-Wood 76ee3060d1 s3: Add MD5 metadata to objects uploaded with SSE-AWS/SSE-C 
Before this change, small objects uploaded with SSE-AWS/SSE-C would
not have MD5 sums.

This change adds metadata for these objects in the same way that the
metadata is stored for multipart uploaded objects.

See: #1824 #2827
 2020-11-25 12:28:02 +00:00
Nick Craig-Wood 4bb241c435 s3: store md5 in the Object rather than the ETag 
This enables us to set the md5 to cache it.

See: #1824 #2827
 2020-11-25 12:28:02 +00:00
Nick Craig-Wood a06f4c2514 s3: fix hashes on small files with aws:kms and sse-c 
If rclone is configured for server side encryption - either aws:kms or
sse-c (but not sse-s3) then don't treat the ETags returned on objects
as MD5 hashes.

This fixes being able to upload small files.

Fixes #1824
 2020-11-25 12:28:02 +00:00
Nick Craig-Wood 53aa03cc44 s3: complete sse-c implementation 
This now can complete all operations with SSE-C enabled.

Fixes #2827
See: https://forum.rclone.org/t/issues-with-aws-s3-sse-c-getting-strange-log-entries-and-errors/20553
 2020-11-25 12:28:02 +00:00
Nick Craig-Wood f0905499e3 random: seed math/rand in one place with crypto strong seed #4783 
This shouldn't be read as encouraging the use of math/rand instead of
crypto/rand in security sensitive contexts, rather as a safer default
if that does happen by accident.
 2020-11-18 17:48:44 +00:00
Nick Craig-Wood 095c7bd801 sharefile: fix backend due to API swapping integers for strings 
For some reason the API started returning some integers as strings in
JSON. This is probably OK in Javascript but it upsets Go.

This is easily fixed with the `json:"name,size"` struct tag.
 2020-11-13 14:37:43 +00:00
Nick Craig-Wood 23469c9c7c ftp: add --ftp-disable-msld option to ignore MLSD for really old servers 
This is useful for servers which advertise MLSD (eg some versions of
Serv-U) but don't support it properly.

See: https://forum.rclone.org/t/double-folder-names-on-target-destination-paths-ftp/18822
See: https://github.com/jlaffaye/ftp/pull/196
 2020-11-13 11:25:34 +00:00
buengese 636fb5344a drive: implement CleanUp workaround for team drives - fixes #2418 2020-11-13 03:30:28 +01:00
buengese bc4282e49e compress: added experimental compression remote - implements #2098, #1356, #675 
This remote implements transparent compression using gzip. Uses JSON as a for storing metadata.

Co-authored-by: id01 <gaviniboom@gmail.com>
 2020-11-13 02:31:59 +01:00
Manish Gupta 95d0410baa
local: continue listing files/folders when a circular symlink is detected 
Before this change a circular symlink would cause rclone to error out from the listings.

After this change rclone will skip a circular symlink and carry on the listing,
producing an error at the end.

Fixes #4743
 2020-11-12 11:32:55 +00:00
Nick Craig-Wood f7efce594b config: add context.Context #3257 #4685 
This add config to the Config callback in the backends and the related
config functions.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood 1fb6ad700f accounting: add context.Context #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood 8b96933e58 fs: Add context to fs.Features.Fill & fs.Features.Mask #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood d846210978 fs: Add context to NewFs #3257 #4685 
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
 2020-11-09 18:05:54 +00:00
Nick Craig-Wood bedf6e90d2 onedrive: warn on gateway timeout errors 
It seems that when doing chunked uploads to onedrive, if the chunks
take more than 3 minutes or so to upload then they may timeout with
error 504 Gateway Timeout.

This change produces an error (just once) suggesting lowering
`--onedrive-chunk-size` or decreasing `--transfers`.

This is easy to replicate with:

    rclone copy -Pvv --bwlimit 0.05M 20M onedrive:20M

See: https://forum.rclone.org/t/default-onedrive-chunk-size-does-not-work/20010/
 2020-11-02 16:53:35 +00:00
Nick Craig-Wood 1973fc1ecc azureblob: update lib from v0.10.0 to v0.11.0 and fix API breakage 
See: https://github.com/Azure/azure-storage-blob-go/issues/226
 2020-10-29 13:34:39 +00:00
Josh Soref 0a6196716c docs: style: avoid double-nesting parens 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref a15f50254a docs: grammar: if, then 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref 5d4f77a022 docs: grammar: Oxford comma 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref a089de0964 docs: grammar: uncountable: links 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref 3068ae8447 docs: grammar: count agreement: files 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref 67ff153b0c docs: grammar: article: a-file 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref 3e1cb8302a docs: spelling: etc. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref d4f38d45a5 docs: spelling: high-speed 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref bbe7eb35f1 docs: spelling: server-side 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
edwardxml 87e54f2dde
ftp: update wording for flags 
Minor wording change to help for explicit and implicit FTPS flags. More consistent between flags. Add 's' to request because only one 'client' mentioned.
 2020-10-28 15:45:52 +00:00
Ivan Andreev be6a888e50 chunker: skip long local hashing, hash in-transit (fixes #4021) 
PR 4614
 2020-10-26 20:18:07 +03:00
Ivan Andreev dad8447423 mailru: avoid prehashing of large local files 
PR 4617
 2020-10-26 20:16:52 +03:00
Ivan Andreev 65ff109065 mailru: accept special folders eg camera-upload 
Fixes #4025
PR 4690
 2020-10-26 20:04:31 +03:00
Nick Craig-Wood cf0bdad5de union: create root directories if none exist 
This fixes the TestUnion: integration test if the /tmp/union[123] dirs
don't exist.
 2020-10-25 18:10:49 +00:00
albertony ffdd0719e7 jottacloud: avoid double url escaping of device/mountpoint - fixes #4697 2020-10-20 17:43:49 +02:00
Anagh Kumar Baranwal 5b09599a23 drive: Added flag `--drive-stop-on-download-limit` to stop transfers when the download limit is exceeded 
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-10-14 15:51:01 +01:00
Josh Soref d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Anagh Kumar Baranwal fc5b14b620
s3: Added `--s3-disable-http2` to disable http/2 
Fixes #4673

Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
 2020-10-13 17:11:22 +01:00
Stephen Harris bbddadbd04 sftp: remember entered password in AskPass mode 
As reported in

  https://github.com/rclone/rclone/issues/4660#issuecomment-705502792

After switching to a password callback function, if the ssh connection
aborts and needs to be reconnected then the user is-reprompted for their
password.  Instead we now remember the password they entered and just give
that back.  We do lose the ability for them to correct mistakes, but that's
the situation from before switching to callbacks.  We keep the benefits
of not asking for passwords until the SSH connection succeeds (right
known_hosts entry, for example).

This required a small refactor of how `f := &Fs{}` was built, so we can
store the saved password in the Fs object
 2020-10-13 16:53:11 +01:00
Nick Craig-Wood 7428e47ebc local: fix sizes and syncing with --links option on Windows - fixes #4581 
Before this change rclone returned the size from the Stat call of the
link. On Windows this reads as 0 always, however on unix it reads as
the length of the text in the link. This caused errors like this when
syncing:

    Failed to copy: corrupted on transfer: sizes differ 0 vs 13

This change causes Windows platforms to read the link and use that as
the size of the link instead of 0 which fixes the problem.
 2020-10-13 16:29:56 +01:00
Dan Hipschman 70f92fd6b3 crypt: small simplification, no functionality change 2020-10-12 17:20:39 +01:00
Nick Craig-Wood 0906f8dd3b onedrive: fix disk usage for sharepoint 
Some onedrive sharepoints appear to return all 0s for quota

    "quota":{"deleted":0,"remaining":0,"total":0,"used":0}

This commit detects this and returns unknown for all quota parts.

See: https://forum.rclone.org/t/zero-size-volume-when-mounting-onedrive-sharepoint/19597
 2020-10-09 14:11:56 +01:00
buengese 664213cedb jottacloud: remove clientSecret from config when upgrading to token based authentication - #4645 2020-10-08 11:51:17 +02:00