The "rcloneremotename" (aka "target") config parameter is now permitted
to contain (1) remote names that are defined by environment variables,
but not in an rclone config file, and (2) backend strings such as
":memory:".
This should fix some of the failing integration tests. For context:
https://github.com/rclone/rclone/pull/7987#issuecomment-2688580667
Issue #7984
Some libraries use `application/json; charset=utf-8` as their `Content-Type`, which is valid.
However we were not decoding the JSON body in that case, resulting in issues communicating with the rcserver.
This commit modernizes Go usage. This was done with:
go run golang.org/x/tools/gopls/internal/analysis/modernize/cmd/modernize@latest -fix -test ./...
Then files needed to be `go fmt`ed and a few comments needed to be
restored.
The modernizations include replacing
- if/else conditional assignment by a call to the built-in min or max functions added in go1.21
- sort.Slice(x, func(i, j int) bool) { return s[i] < s[j] } by a call to slices.Sort(s), added in go1.21
- interface{} by the 'any' type added in go1.18
- append([]T(nil), s...) by slices.Clone(s) or slices.Concat(s), added in go1.21
- loop around an m[k]=v map update by a call to one of the Collect, Copy, Clone, or Insert functions from the maps package, added in go1.21
- []byte(fmt.Sprintf...) by fmt.Appendf(nil, ...), added in go1.19
- append(s[:i], s[i+1]...) by slices.Delete(s, i, i+1), added in go1.21
- a 3-clause for i := 0; i < n; i++ {} loop by for i := range n {}, added in go1.22
SSH servers which implement file transfer protocols are vulnerable to
a denial of service attack from clients which complete the key
exchange slowly, or not at all, causing pending content to be read
into memory, but never transmitted.
This affects users of `rclone serve sftp`.
See: https://pkg.go.dev/vuln/GO-2025-3487
This commit registers gitannex's unit tests with the integration tester
by updating the config.yaml file.
Since we have not yet updated the e2e tests to use the fstest framework,
this commit also adds a case to the e2e tests' skipE2eTestIfNecessary()
function.
Issue #7984
This enables the unit tests to run on any given backend, via the
`-remote` flag, e.g. `go test -v ./cmd/gitannex/... -remote dropbox:`.
We should also port the gitannex e2e tests at some point.
Issue: #7984
In this commit
ceef78ce44 vfs: fix directory cache serving stale data
We added a new test which caused lots of integration test failures.
This fixes the problem by disabling the test unless the feature flag
DirModTimeUpdatesOnWrite is present on the remote.
Sometimes the Azure blob servers reply with 503 ServerBusy errors and
these should be retried.
Before this change, when testing to see if a single part server side
copy was done, ServerBusy errors were returned to the user rather than
being retried.
Wrapping the call in the pacer fixes the problem and ensures it is
retried properly using the --low-level-retries mechanism.
Before this change, when executed on a directory, rclone would only
touch files sequentially.
This change makes rclone touch --transfers files at once.
Fixes#8402
Found via `codespell -q 3 -S "./docs/static,./fs/rc/params_test.go" -L aadd,afile,alledges,bbefore,bu,buda,copys,couldn,crashers,crypted,ddelete,deriver,failre,goup,hashin,hel,inbraces,keep-alives,ket,medias,ment,mis,nd,nin,notin,ois,ot,parth,re-use,re-using,responser,rin,sav,splited,streamin,synching,te,twoo,ue,unknwon,wasn`
These files must be "exported" to be useful. The export process
is controlled by the --dropbox-export-formats flag and the ancilliary flags
--dropbox-skip-exports and --dropbox-show-all-exports modeled on the
Google drive equivalents
The VFS directory cache layer didn't update directory entry properties
if they are reused after cache invalidation.
Update them unconditionally as newDir sets them to the same value and
setting a pointer is cheaper in both LoC as well as CPU cycles than a
branch.
Also add a test exercising this behavior.
Fixes#6335
Before this change, if --vfs-cache-mode writes or above was set and
--links was in use, when a symlink was saved then the VFS failed to
upload it. This meant when the VFS was restarted the link wasn't there
any more.
This was caused by the local backend, which we use to manage the VFS
cache, picking up the global --links flag.
This patch makes sure that the internal instantations of the local
backend in the VFS cache don't ever use the --links flag or the
--local-links flag even if specified on the command line.
Fixes#8367
Before, after a sync, only file modtimes were updated when not using
--copy-empty-src-dirs. This ensures modtimes are updated to match the source
folder, regardless of copyEmptySrcDir. The flag --no-update-dir-modtime
(which previously did nothing) will disable this.
This adds tests to check dir modtimes are updated from source
when syncing even if they've changed in the destination.
This should work both with and without --copy-empty-src-dirs.
5f70918e2c
introduced a new INFO log when making a directory, which differs depending on
whether the backend supports setting directory metadata. This caused false
positives on the bisync createemptysrcdirs test.
This fixes it by ignoring that log line.
This shifts the behavior of the average loop to be a persistent loop
that gets resumed/paused when transfers & checks are started/completed.
Previously, the averageLoop was stopped on completion of
transfers & checks but failed to start again due to the protection of
the sync.Once
Signed-off-by: Anagh Kumar Baranwal <6824881+darthShadow@users.noreply.github.com>
Before this change, there was a bug affecting listing files when:
- a given bisync run had changes in the 2to1 direction
AND
- the run had NO changes in the 1to2 direction
AND
- at least one of the changed files changed AGAIN during the run
(specifically, after the initial march and before the transfers.)
In this situation, the listings on one side would still retain the prior version
of the changed file, potentially causing conflicts or errors.
This change fixes the issue by making sure that if we're updating the listings
on one side, we must also update the other. (We previously tried to skip it for
efficiency, but this failed to account for the possibility that a changed file
could change again during the run.)
Nick Craig-Wood
Dan McArdle
Zachary Vorhies
Jonathan Giannuzzi
luzpaz
Dave Vasilevsky
Dave Vasilevsky
emyarod
Lorenz Brun
Anagh Kumar Baranwal
Michael Kebe
jbagwell-akamai
Janne Hellsten
ll3006
nielash
Joel K Biju