7 Commits

Author	SHA1	Message	Date
Nick Craig-Wood	401cf81034	build: modernize Go usage ... This commit modernizes Go usage. This was done with: go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./... Then files needed to be `go fmt`ed and a few comments needed to be restored. The modernizations include replacing - if/else conditional assignment by a call to the built-in min or max functions added in go1.21 - sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21 - interface{} by the 'any' type added in go1.18 - append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21 - loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21 - []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19 - append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21 - a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22	2025-02-28 11:31:14 +00:00
Nick Craig-Wood	1317fdb9b8	build: fix comments after golangci-lint upgrade	2024-11-11 18:03:36 +00:00
Nick Craig-Wood	e686e34f89	multipart: make pool buffer size public	2024-08-14 21:13:09 +01:00
Nick Craig-Wood	2db0e23584	backends: change OpenChunkWriter interface to allow backend concurrency override ... Before this change the concurrency used for an upload was rather inconsistent. - if size below `--backend-upload-cutoff` (default 200M) do single part upload. - if size below `--multi-thread-cutoff` (default 256M) or using streaming uploads (eg `rclone rcat) do multipart upload using `--backend-upload-concurrency` to set the concurrency used by the uploader. - otherwise do multipart upload using `--multi-thread-streams` to set the concurrency. This change makes the default for the concurrency used be the `--backend-upload-concurrency`. If `--multi-thread-streams` is set and larger than the `--backend-upload-concurrency` then that will be used instead. This means that if the user sets `--backend-upload-concurrency` then it will be obeyed for all multipart/multi-thread transfers and the user can override them all with `--multi-thread-streams`. See: #7056	2023-09-03 11:47:05 +01:00
Nick Craig-Wood	e953598987	build: fix lint errors when re-enabling revive exported & package-comments	2023-08-29 13:03:13 +01:00
Nick Craig-Wood	967fc6d7f4	lib/multipart: fix accounting for multipart transfers ... This change makes sure the accouting is done when bytes are taken out of the buffer rather than put in. See: https://forum.rclone.org/t/improve-transfer-stats-calculation-for-multipart-uploads/41172	2023-08-27 23:10:58 +01:00
Nick Craig-Wood	4c76fac594	s3: factor generic multipart upload into lib/multipart #7056 ... This makes the memory controls of the s3 backend inoperative and replaced with the global ones. --s3-memory-pool-flush-time --s3-memory-pool-use-mmap By using the buffered reader this fixes excessive memory use when uploading large files as it will share memory pages between all readers. Fixes #7141	2023-08-24 12:39:27 +01:00