Commit Graph

3493 Commits

Author SHA1 Message Date
Aleksandar Jankovic 53a1a0e3ef accounting: add reference to completed transfers
Add core/transferred call that lists completed transfers and their
status.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic 8243ff8bc8 accounting: isolate stats to groups
Introduce stats groups that will isolate accounting for logically
different transferring operations. That way multiple accounting
operations can be done in parallel without interfering with each other
stats.

Using groups is optional. There is dedicated global stats that will be
used by default if no group is specified. This is operating mode for CLI
usage which is just fire and forget operation.

For running rclone as rc http server each request will create it's own
group. Also there is an option to specify your own group.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic be0464f5f1 accounting: change stats interface
This is done to make clear ownership over accounting object and prepare
for removing global stats object.

Stats elapsed time calculation has been altered to account for actual
transfer time instead of stats creation time.
2019-07-28 14:48:19 +01:00
Nick Craig-Wood 2d561b51db Add EliEron to contributors 2019-07-28 12:33:21 +01:00
Nick Craig-Wood 9241a93c2d Add justinalin to contributors 2019-07-28 12:33:21 +01:00
EliEron fb32f77bac Docs: Fix typos in filtering demonstrations 2019-07-28 12:32:53 +01:00
justinalin 520fb03bfd log: add --use-json-log for JSON logging 2019-07-28 12:05:50 +01:00
justinalin a3449bda30 vendor: add github.com/sirupsen/logrus 2019-07-28 12:05:50 +01:00
yparitcher ccc416e62b b2: Fix link sharing #3314 2019-07-28 11:47:31 +01:00
jaKa a35aa1360e Support setting modification times on Koofr backend.
Configuration time option to disable the above for if using Dropbox (does not
allow setting mtime on copy) or Amazon Drive (neither on upload nor on copy).
2019-07-24 21:11:58 +01:00
jaKa 3df9dbf887 vendor: updated github.com/koofr/go-koofrclient for set mtime support. 2019-07-24 21:11:58 +01:00
Nick Craig-Wood 9af0a704af Add Paul Millar to contributors 2019-07-24 20:34:39 +01:00
Nick Craig-Wood 691e5ae5f0 Add Yi FU to contributors 2019-07-24 20:34:39 +01:00
Nick Craig-Wood 5a44bafa4e fstest: add fs.ErrorCantShareDirectories for backends which can only share files 2019-07-24 20:34:29 +01:00
Nick Craig-Wood 8fdce31700 config: Fix hiding of options from the configurator 2019-07-24 20:34:29 +01:00
Nick Craig-Wood 493dfb68fd opendrive: refactor to use existing lib/rest facilities for uploads
This also checks the return of the call to make sure the number of
bytes written was as expected.
2019-07-24 20:34:29 +01:00
Nick Craig-Wood 71587344c6 lib/rest: allow Form upload with no file to upload 2019-07-24 20:34:29 +01:00
yparitcher 8e8b78d7e5 Implement --compare-dest & --copy-dest Fixes #3278 2019-07-22 19:42:29 +01:00
Nick Craig-Wood 266600dba7 build: reduce parallelism in cross compile to reduce memory and fix Travis
Before this change Travis builds were running out of memory when cross
compiling all the OSes.
2019-07-22 17:10:26 +01:00
Paul Millar e4f6ccbff2 webdav: add docs for using bearer_token_command with oidc-agent 2019-07-22 16:01:55 +01:00
Nick Craig-Wood 1f1ab179a6 webdav: refresh token when it expires with --webdav-bearer-token-command
Fixes #2380
2019-07-22 16:01:55 +01:00
Nick Craig-Wood c642531a1e webdav: add --webdav-bearer-token-command - fixes #2380
This can be used with oidc-agent to get a bearer token
2019-07-22 15:59:54 +01:00
buengese 19ae053168 rcserver: remove _async key from input parameters after parsing so later operations won't get confused - fixes #3346 2019-07-20 19:35:10 +02:00
buengese def790986c fichier: make FolderID int and adjust related code - fixes #3359 2019-07-20 02:49:08 +02:00
Yi FU 0a1169e659 ssh: opt-in support for diffie-hellman-group-exchange-sha256 diffie-hellman-group-exchange-sha1 - fixes #1810 2019-07-13 12:21:56 +02:00
Nick Craig-Wood 5433021e8b drive: fix server side copy of big files
Before this change rclone was sending a MimeType in the requests for
server side Move and Copy.

The conjecture is that if you attempt to set the MimeType to something
different in a Copy then Google Drive has to do an actual copy of the
file data.  This takes a very long time (since it is large) and fails
after a 90s timeout.

After the change we no longer set the MimeType in Move or Copy and the
copies happen instantly and correctly.

Many thanks to @darthShadow for discovering that this was causing the
problem.

Fixes #3070
Fixes #3033
Fixes #3300
Fixes #3155
2019-07-05 10:49:19 +01:00
Nick Craig-Wood c9f77719e4 b2: enable server side copy to copy between buckets - fixes #3303 2019-07-05 10:07:05 +01:00
Nick Craig-Wood 3cd63a00be googlephotos: fix configuration walkthrough 2019-07-04 15:19:59 +01:00
Nick Craig-Wood d7016866e0 googlephotos: fix creation of duplicated albums
Also make sure we don't list the albums twice
2019-07-04 13:45:52 +01:00
yparitcher d72e4105fb b2: Fix link sharing #3314 2019-07-04 11:53:59 +01:00
yparitcher b4266da4eb sync: fix SyncSuffix tests #3272 2019-07-03 17:36:22 +01:00
yparitcher 3f5767b94e b2: Implement link sharing #2178 2019-07-03 14:10:25 +01:00
Nick Craig-Wood 1510e12659 build: move race test to go modules build
In an attempt to even out the build times.
2019-07-03 12:07:29 +01:00
Nick Craig-Wood ede03258bc build: use go modules proxy when building modules 2019-07-03 12:07:29 +01:00
Nick Craig-Wood 7fcbb47b1c build: split other OS build into a separate builder
This is in order to make longest build (the Linux build) quicker
2019-07-03 12:07:29 +01:00
Nick Craig-Wood 9cafeeb4b6 dirtree: make tests more reliable 2019-07-02 16:29:40 +01:00
Nick Craig-Wood a1cfe61ffd googlephotos: Backend for accessing Google Photos #369 2019-07-02 15:26:55 +01:00
Nick Craig-Wood 5eebbaaac4 test_all: add `tests` parameter to limit which tests to run for a backend 2019-07-02 15:26:55 +01:00
Nick Craig-Wood bc70bff125 fs/dirtree: factor DirTree out of fs/walk and add tests 2019-07-02 15:26:55 +01:00
Nick Craig-Wood cf15b88efa build: make explicit which matrix items we will deploy 2019-07-02 14:13:46 +01:00
Nick Craig-Wood dcaee0016a build: add a builder for building with go modules 2019-07-02 12:33:03 +01:00
Nick Craig-Wood 387b496d1e operations: fix tests TestMoveFileBackupDir and TestCopyFileBackupDir again
Commit 734f504d5f wasn't tested properly and had a typo which
caused it not to build :-(
2019-07-02 12:22:29 +01:00
Nick Craig-Wood 734f504d5f operations: fix tests TestMoveFileBackupDir and TestCopyFileBackupDir
..so they don't run on backends which can't move or copy.
2019-07-02 10:46:49 +01:00
Dan Dascalescu 7153909390 docs: fix typo in google drive docs 2019-07-02 10:10:08 +01:00
Russell Davis ea35e807db docs: clarify `--update` and `--use-server-mod-time`
It's likely a mistake to use `--use-server-modtime` if you're not also using `--update`. It might even make sense to emit a warning in the code when doing this, but for now, I made it more clear in the docs.

I also clarified how `--use-server-modtime` can be useful in the `--update` section.
2019-07-02 10:09:03 +01:00
Nick Craig-Wood 5df5a3b78e vendor: tidy go.mod and go.sum - fixes #3317 2019-07-02 09:47:00 +01:00
Nick Craig-Wood 37c1144b46 Add Russell Davis to contributors 2019-07-02 07:57:18 +01:00
Nick Craig-Wood 8d116ba0c9 Add Matti Niemenmaa to contributors 2019-07-02 07:57:18 +01:00
Russell Davis 6a3c3d9b89 Update docs on S3 policy to include ListAllMyBuckets permission
This permission is required for `rclone lsd`.
2019-07-02 07:56:54 +01:00
Matti Niemenmaa a6dca4c13f s3: Add INTELLIGENT_TIERING storage class
For Intelligent-Tiering:
https://aws.amazon.com/s3/storage-classes/#Unknown_or_changing_access
2019-07-01 18:17:48 +01:00