MirrorSave
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,469 Commits 14 Branches 49 Tags 90 MiB
Commit Graph

586 Commits

Author SHA1 Message Date
streambinder 97e5ce4344 internal: backend: sftp: support user@domain parsing as user 2019-12-19 13:15:37 +01:00
rawtaz da4473aba6
Merge pull request #2391 from kastenhq/refresh-lock-time 
Update Lock.Time in lock.Refresh()
 2019-11-26 21:44:29 +01:00
rawtaz 6e85a58045
Merge pull request #2465 from whs-dot-hk/fix-test-metadata-changed 
Fix test metadata changed
 2019-11-26 21:42:42 +01:00
rawtaz e14c4b1737
Merge pull request #2484 from restic/add-s3-region 
s3: Allow specifying region
 2019-11-22 15:51:17 +01:00
Alexander Neumann 409909a7f5 Add option description for Region 2019-11-22 15:09:09 +01:00
mdauphin df500a372d Add AWS_REGION env var to specify s3 region 2019-11-22 15:04:48 +01:00
Alexander Neumann a6e8af7e0f Update minio-go 2019-11-22 14:50:46 +01:00
rawtaz 41fe9318b1
Merge pull request #2425 from thiell/restic_cache_dir_env 
Add support for $RESTIC_CACHE_DIR
 2019-11-20 21:18:57 +01:00
Leo R. Lundgren 4d2aa18273 Accept both singular and plural for --group-by (#2330) 2019-11-20 19:25:48 +01:00
rawtaz 26d1f9f4ba
Merge pull request #2307 from gary-kim/fix-2306-password-retries 
Allow multiple retries for interactive password input
 2019-11-20 18:30:20 +01:00
whs 7cacba0394 Assume WithAtime default to false 2019-11-06 16:38:46 +08:00
Stephane Thiell 0e897ef7b8 Add support for $RESTIC_CACHE_DIR 
Add support for restic-specific $RESTIC_CACHE_DIR environment variable
to override the cache directory like --cache-dir would have.
 2019-09-26 15:59:56 -07:00
Eric Hamilton 1596d06f8e Update Lock.Time in lock.Refresh() 2019-09-04 11:38:35 -07:00
Vladimir Rusinov db20c0b8d0 Ignore username difference in TestMetadataChanged. 
In some (rare) cases "fake" UID 51234 may exist in a system running a
test. When this is the case, `cmp.Equal(want, node3)` will fail based on
difference between empty string and an actual username present in a
system.

Fixes github issue #2372
 2019-08-13 22:25:00 +01:00
Alexander Neumann 01c51b3449 Merge pull request #2321 from restic/fix-index-error-handling 
Check errors returned by LoadIndex()
 2019-07-27 09:37:18 +02:00
Alexandr Bruyako e19622e4b1 start using fileMode const 2019-07-01 00:26:00 +03:00
Alexandr Bruyako 38ea7ed4f6 remove unused code 2019-07-01 00:24:45 +03:00
Alexandr Bruyako 76d1866444 avoiding unnecessary type conversions 2019-06-30 23:58:00 +03:00
Alexandr Bruyako 8b22fe29cf improved slice copying 2019-06-30 23:56:36 +03:00
Alexandr Bruyako 02014be76c simplified prefix removal, removed unnecessary if-else statements 2019-06-30 23:34:47 +03:00
Alexandr Bruyako 16eeed2ad5 simplified string sorting by using a more suitable function 2019-06-30 23:20:32 +03:00
Alexander Neumann 88716794e3 Check errors returned by LoadIndex() 
Bug was reported in the forum here: https://forum.restic.net/t/check-rebuild-index-prune/1848/13
 2019-06-30 21:34:53 +02:00
Gary Kim fea2464d4d
Allow multiple retries for interactive password input 
Restic used to quit if the repository password was typed incorrectly once.
Restic will now ask the user again for the repository password if typed incorrectly.
The user will now get three tries to input the correct password before restic quits.
 2019-06-13 20:11:02 +08:00
Garry McNulty 8066195e6e fs: Handle absolute pathname for --stdin-filename 
Return valid directory info from Lstat() for parent directories of the
specified filename. Previously only "/" and "." were valid directories.

Also set directory mode as this is checked by archiver.

Closes #2063
 2019-05-08 15:21:27 +02:00
Garry McNulty f7f14cf8c9 fs: Add file info base name check in reader tests (#2063) 2019-05-08 15:21:27 +02:00
Garry McNulty 5096f3b491 fs: Update directory check in reader tests (#2063) 2019-05-08 15:21:27 +02:00
Alexander Neumann 920d458a4a archiver: Use untyped constants for testing FileInfo 2019-05-05 14:57:38 +02:00
Alexander Neumann b016dc2ff0 archiver/Windows: Skip test new-content-same-filestamp 2019-05-05 14:02:11 +02:00
Alexander Neumann 355db0bc29 windows: Use LastWriteTime for ctime and mtime 
Windows does not have a concept of a `change time` in the sense as Unix
has it: the field `CreationTime` of the `Win32FileAttributeData` struct
is not updated when attributes or content is changed. So from now on
we're using the `LastWriteTime` as the `change time` on Windows.
 2019-05-05 14:02:11 +02:00
Alexander Neumann 6e2fe73189 archiver: Move tests back into the same file 
Move all Archiver tests back into `archiver_test.go` and add some tiny
helpers to mock what `lstat` returns (for Windows and Unix separately).
 2019-05-05 14:02:11 +02:00
Alexander Neumann 303a5dab6a archiver: Clarify value in test struct 
Since I could not remember what the value for `Check` means this commit
renames it to `SameFile`: when set to true, the test should make sure
that `FileChanged` should return false (=file is unmodified).
 2019-05-05 12:57:00 +02:00
Gábor Lipták bce6438d22
Don't run TestMetadataChanged test on Windows 
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
 2019-04-27 21:23:47 -04:00
Alexander Neumann 919dd2ac84 Merge pull request #2252 from restic/fix-2249 
Read fresh metadata for unmodified files
 2019-04-25 09:15:50 +02:00
Alexander Neumann 870bc5108e Merge pull request #2253 from restic/fix-2174 
Make sure timestamps are valid
 2019-04-25 09:14:43 +02:00
Alexander Neumann a6481b3707 Merge pull request #2212 from cbane/check-ctime 
Examine file ctime when checking if files have changed.
 2019-04-25 08:58:27 +02:00
Courtney Bane 0ebfc55ee3 Use existing setTimestamp function for ctime test and improve error checking. 2019-04-24 20:53:08 -05:00
Courtney Bane 35b7607802 Don't check ctime when ignoring inode. 2019-04-24 20:53:08 -05:00
Alexander Neumann 939f3e972c node: Make sure year of all timestamps is valid 
Sometimes restic gets bogus timestamps which cannot be converted to
JSON, because the stdlib JSON encoder returns an error if the year is
not within [0, 9999]. We now make sure that we at least record _some_
timestamp and cap the year either to 0000 or 9999. Before, restic would
refuse to save the file at all, so this improves the status quo.

This fixes #2174 and #1173
 2019-04-24 16:39:36 +02:00
Alexander Neumann 389067fb8b Only use list of blobs for old node 
Closes #2249
 2019-04-24 15:07:26 +02:00
Alexander Neumann 4b0ca9ddab Add test for #2249 2019-04-24 15:07:23 +02:00
Courtney Bane b8c2544dcb Examine file ctime when checking if files have changed. 2019-04-23 21:54:35 -05:00
Alexander Neumann 18ec27a0da Merge pull request #2243 from treymo/ls-length-2104 
backend find: Check prefix length < snapshot IDs when searching
 2019-04-23 09:41:19 +02:00
Jan Niklas Richter 733519d895 Refactor duplicate code for grouping snapshots 
This commit is a followup to the addition of the --group-by flag for the
snapshots command. Adding the grouping code there introduced duplicated
code (the forget command also does grouping). This commit refactors
boths sides to only use shared code.
 2019-04-22 14:57:53 +02:00
Trey Moore 5ac9c1157a Prevent "slice bounds out of range" error if prefix is longer than snapshot IDs. 
This includes tests as well as other tests for the backend_find.Find function. Issue #2104.
 2019-04-16 20:26:20 -07:00
Alexander Neumann 6b700d02f5 Merge pull request #2217 from restic/improve-memory-usage 
WIP: improve memory usage
 2019-04-13 15:07:07 +02:00
Alexander Neumann 7304738872 check: Reduce default parallelism from 40 to 5 2019-04-13 13:38:39 +02:00
Alexander Neumann 66efa425bf Reuse buffer in worker functions 2019-04-13 13:38:39 +02:00
Alexander Neumann d51e9d1b98 Add []byte to repo.LoadAndDecrypt and utils.LoadAll 
This commit changes the signatures for repository.LoadAndDecrypt and
utils.LoadAll to allow passing in a []byte as the buffer to use. This
buffer is enlarged as needed, and returned back to the caller for
further use.

In later commits, this allows reducing allocations by reusing a buffer
for multiple calls, e.g. in a worker function.
 2019-04-13 13:38:39 +02:00
Alexander Neumann e046428c94 Replace FilesInParallel with an errgroup.Group 2019-04-13 13:38:39 +02:00
Alexander Neumann 75906edef5 mock: Remove unused repository type 2019-04-13 13:38:39 +02:00
Alexander Neumann 203d775190 restic: Make JSON unmarshal for ID more efficient 
This commit reduces several allocations in UnmarshalJSON() by decoding
the hex string directly in a single step.
 2019-04-13 13:38:39 +02:00
Benoît Knecht 3c112d9cae s3: Add config option to set storage class 
The `s3.storage-class` option can be passed to restic (using `-o`) to
specify the storage class to be used for S3 objects created by restic.

The storage class is passed as-is to S3, so it needs to be understood by
the API. On AWS, it can be one of `STANDARD`, `STANDARD_IA`,
`ONEZONE_IA`, `INTELLIGENT_TIERING` and `REDUCED_REDUNDANCY`. If
unspecified, the default storage class is used (`STANDARD` on AWS).

You can mix storage classes in the same bucket, and the setting isn't
stored in the restic repository, so be sure to specify it with each
command that writes to S3.

Closes #706
 2019-03-26 16:37:07 +01:00
Alexander Neumann c1058005c3 Fix build on FreeBSD for Go < 1.12 2019-03-16 20:52:37 +01:00
Alexander Neumann ca73808649 Merge pull request #2197 from dmgk/master 
Fix build on FreeBSD with Go 1.12
 2019-03-16 20:23:19 +01:00
Igor Fedorenko bf9a507148 Fix rare 'file already closed' during restore 
Fixes #2183

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2019-03-16 17:01:20 +01:00
Alexander Neumann 65b476ead9 Fix gofmt 2019-03-16 13:29:05 +01:00
Alexander Neumann aaa1cc2c26 Merge pull request 2193 from restic/allow-empty-rclone-args 
rclone: Rework backend option parsing
 2019-03-16 12:17:38 +01:00
Alexander Neumann 95434cff16 Merge pull request 2156 from kayrus/swift-app-cred 
Swift: introduce application credential auth support
 2019-03-16 12:17:17 +01:00
Heiko Bornholdt db8f5864fc Add --ignore-inode option to backup cmd 
revised version of https://github.com/restic/restic/pull/2047
 2019-03-10 21:24:29 +01:00
Dmitri Goutnik 1d8b21cdad Fix build on FreeBSD with Go 1.12 2019-03-06 09:41:49 -05:00
Alexander Neumann 3865b59716 rclone: Rework backend option parsing 
This change allows passing no arguments to rclone, using `-o
rclone.args=""`. It is helpful when running rclone remotely via SSH
using a key with a forced command (via `command=` in `authorized_keys`).
 2019-03-02 10:36:42 +01:00
Alexander Neumann e277a92a2f self-update: Don't cancel download after 30 seconds 2019-02-23 11:15:18 +01:00
kayrus 6ebcfe7c18 Swift: introduce application credential auth support 2019-02-14 14:19:05 +01:00
Alexander Neumann fed25714a4 Merge pull request 2139 from restic/fail-zero-bytes-stdin 
Return error when reading zero byte from stdin
 2019-02-10 12:34:05 +01:00
Alexander Neumann 8906d85ab8 Merge pull request 2138 from restic/fix-2136 
Use processed bytes for summary
 2019-02-10 12:32:50 +01:00
Matthew Holt 6cf13483b5 Add snapshot ID to summary output 2019-02-10 12:09:09 +01:00
Matthew Holt f645306a18 Initial implementation of JSON progress reporter 2019-02-10 12:09:09 +01:00
Alexander Neumann 1f246c5309 Merge pull request 2090 from plumbeo/within-hours 
Increase the granularity of the "keep within" snapshot retention policy
 2019-01-06 15:04:55 +01:00
Alexander Neumann c4fbf2c779 Return error when reading zero byte from stdin 
This commit changes the internal file system implementation for reading
data from stdin, it now returns an error when no bytes could be read. I
think it's worth failing in this case, the user instructed restic to
read some data from stdin, and no data was read at all. Maybe it was in
a pipe and some earlier stage failed.

See #2135 for a short discussion.
 2019-01-06 14:52:20 +01:00
Alexander Neumann 7c084014fa Use processed bytes for summary 
When restic reads the backup from stdin, the number of bytes processed
was always displayed as zero. The reason is that the UI for the archive
uses the total bytes as returned by the scanner, which is zero for
stdin. So instead we keep track of the real number of bytes processed
and print that at the end.

Closes #2136
 2019-01-06 13:15:23 +01:00
Alexander Neumann 98526b8dbe Merge pull request 2100 from restic/fix-1989 
gs: Respect bandwidth limiting
 2018-11-28 21:39:42 +01:00
Alexander Neumann 0083680d33 Merge pull request 2099 from restic/fix-1229 
cache: Don't return an error for truncated files
 2018-11-28 21:39:35 +01:00
plumbeo 71891b340c Support time ranges expressed in hours in snapshot retention policies 
Make restic forget --keep-within accept time ranges measured in hours and choose
accordingly which snapshots to keep and which to forget. Add relative tests.
 2018-11-26 14:27:42 +01:00
Alexander Neumann c9745cd47e gs: Respect bandwidth limiting 
In 0dfdc11ed9, accidentally we dropped
using the provided http.RoundTripper, this commits adds it back.

Closes #1989
 2018-11-25 18:52:32 +01:00
Chris Howie 1688713400 Add key hinting (#2097) 2018-11-25 09:13:18 -05:00
Alexander Neumann 00597284de cache: Don't return an error for truncated files 
Closes #1229
Closes #1328
 2018-11-25 14:18:02 +01:00
plumbeo 7486bfea5b Extend custom Duration granularity to hours and add tests 2018-11-14 17:27:02 +01:00
Alexander Neumann 5b95bb7059 Fix error message on Windows 2018-11-11 14:06:32 +01:00
Alexander Neumann bfa18ee8ec DownloadAndHash: Check error returned by Load() 2018-10-28 21:28:56 +01:00
Alexander Neumann 0b600d6cef index.New: Test various error conditions 2018-10-28 14:02:31 +01:00
Alexander Neumann 3ae2a79bdf restic: Use local rand source instead of global one 2018-10-28 14:01:30 +01:00
Alexander Neumann f7c0893f76 index: Add tests for error conditions 2018-10-28 13:16:36 +01:00
Alexander Neumann c3de301fc8 index: Use interfaces instead of *repository.Repository 2018-10-28 12:58:34 +01:00
Alexander Neumann 944b446ac0 index: Cancel context for tests 2018-10-28 11:17:04 +01:00
Alexander Neumann b096fc7abf index: Correctly process errors listing all files 
This also removes the now unused `list` and `worker` packages.
 2018-10-28 11:16:29 +01:00
Alexander Neumann 7ac683c360 rclone: Inject debug logger for HTTP 2018-10-21 19:58:40 +02:00
Gábor Lipták e5d7879622
Correct ineffassign 
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
 2018-10-19 16:58:14 -04:00
Alexander Neumann c0fca3f50a restore: Close files tested with --verify 
Before, the target files were opened once per blob and never closed,
this commit fixes that.
 2018-10-14 21:00:14 +02:00
Alexander Neumann 4c2072d875 Merge pull request 1719 from ifedorenko/mt-restorer 
new optimized multithreaded restore implementation
 2018-10-14 20:29:16 +02:00
Igor Fedorenko bda8d7722e restorer: Optimize empty file restore 
don't create fileInfo structs for empty files. this saves memory.
this also avoids extra serial scan of all fileInfo, which should
make restore faster and more consistent.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko c2bcb764cd restorer: Add a note on hardlink metadata 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko 9e24154ec9 restore: significantly reduce memory footprint 
reworked restore error callback to use file location
path instead of much heavier Node. this reduced restore
memory usage by as much as 50% in some of my tests.

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko 9f3ca97ee8 restore: Chang fileInfo to use snapshot location instead of target path 
* uses less memory as common prefix is only stored once
* stepping stone for simpler error callback api, which
  will allow further memory footprint reduction

Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko 32d5ceba87 restore: Fix packcache capacity math with failed downloads 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Igor Fedorenko e010f3b884 restore: Fix small memory leak in filesWriter, add tests 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:39:42 +02:00
Alexander Neumann 941202c119 restore: Rename things, match the rest of the project 2018-10-14 17:39:37 +02:00
Alexander Neumann c021ad2334 restore: Move documentation to godoc 2018-10-14 17:39:35 +02:00
Igor Fedorenko da57302fca restore: Removed legacy restore implementation 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:37:30 +02:00
Igor Fedorenko 1869930d95 restore: New optimized multithreaded implementation 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:37:30 +02:00
Igor Fedorenko 1213d8fef4 Cleanup: more realistic restorer test data setup 
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
 2018-10-14 17:37:30 +02:00