Commit Graph

3398 Commits

Author SHA1 Message Date
Adria Navarro c198a5ae03
Update packages/backend-core/src/redis/redis.ts
Co-authored-by: Sam Rose <hello@samwho.dev>
2024-03-06 12:23:04 +01:00
Adria Navarro 4baadadaa8 Use pipeline instead of eval 2024-03-06 12:22:20 +01:00
Adria Navarro 61c4b83650 Test with spies 2024-03-06 11:55:49 +01:00
Sam Rose 2b206f2105
Fix the TODO I left myself last night. 2024-03-06 10:00:02 +00:00
Adria Navarro d35d382ddb Increase timeouts 2024-03-06 00:58:05 +01:00
Adria Navarro 8e8378d1be Delete info on process 2024-03-06 00:56:30 +01:00
Adria Navarro a9046e9f26 Use lock name 2024-03-06 00:34:48 +01:00
Adria Navarro 3dbf0b3a64 Throw errors on lock not executed 2024-03-06 00:34:48 +01:00
Adria Navarro 0a2fb4a3a6 Use message id to handle concurrency 2024-03-06 00:34:48 +01:00
Adria Navarro 6707da4ac2 Use bulk 2024-03-06 00:34:48 +01:00
Adria Navarro f5e2dc7a27 Do not use lock 2024-03-06 00:34:48 +01:00
Adria Navarro e584d82e6f Improve redlock non executed response 2024-03-06 00:34:48 +01:00
Adria Navarro 37935112a4 Add comment 2024-03-06 00:34:48 +01:00
Adria Navarro c4fc94fb28 Remove defaults and init 2024-03-06 00:34:48 +01:00
Adria Navarro fb2c7467b0 Clean 2024-03-06 00:34:48 +01:00
Adria Navarro caff2876dd Tests 2024-03-06 00:34:48 +01:00
Adria Navarro 151bfd103b Use bull 2024-03-06 00:34:48 +01:00
Adria Navarro e4d8fe2f47 Add doc-writethrough queue 2024-03-06 00:34:48 +01:00
Adria Navarro 770aff4f5e Type inMemoryQueue 2024-03-06 00:34:48 +01:00
Adria Navarro 3c944073eb Use overloads 2024-03-06 00:34:48 +01:00
Adria Navarro eb78103764 Namespace key in redis by db 2024-03-06 00:34:48 +01:00
Adria Navarro 6bd7e6b374 Namespace key in redis by db 2024-03-06 00:34:47 +01:00
Adria Navarro 608865fe9d Fix types 2024-03-06 00:34:47 +01:00
Adria Navarro cd56e1ffff Type caches 2024-03-06 00:34:47 +01:00
Adria Navarro 8062c28722 Making code more readable 2024-03-06 00:34:47 +01:00
Adria Navarro 6a81d21cb7 Fixes and tests 2024-03-06 00:34:47 +01:00
Adria Navarro 00bf88c5bf Extra tests 2024-03-06 00:34:47 +01:00
Adria Navarro 319b3afddb Ensure keys are removed 2024-03-06 00:34:47 +01:00
Adria Navarro e228b9c207 Test concurrency 2024-03-06 00:34:47 +01:00
Adria Navarro b1e1226de6 Add extra tests 2024-03-06 00:34:47 +01:00
Adria Navarro b1027527b3 Add persisting tests 2024-03-06 00:34:47 +01:00
Adria Navarro 64ea969aaf docWritethrough test 2024-03-06 00:34:47 +01:00
Adria Navarro 3998faaf3a DatabaseImpl.docExists test 2024-03-06 00:34:47 +01:00
Adria Navarro 48c92377ad USe get for doc exists 2024-03-06 00:34:47 +01:00
Adria Navarro c96c57bd9e DocWritethrough 2024-03-06 00:34:47 +01:00
Adria Navarro 86b715d5ce Create docWriteThrough redis cache 2024-03-06 00:34:47 +01:00
Adria Navarro 8f486e7bfc Add tests 2024-03-06 00:33:53 +01:00
Adria Navarro 355dea5b05 Add deleteIfValue 2024-03-06 00:33:53 +01:00
Adria Navarro 60f82610f6 Use redis testcontainer 2024-03-06 00:33:24 +01:00
Adria Navarro b774987e61 Install testcontainers 2024-03-06 00:33:04 +01:00
Adria Navarro d4ffc3d5d7 More tests 2024-03-05 22:54:39 +01:00
Adria Navarro 192d7deb2a Handle errors 2024-03-05 22:52:44 +01:00
Adria Navarro 837395e5e0 Add tests 2024-03-05 22:43:34 +01:00
Adria Navarro 339ca403bf Add redis.increment 2024-03-05 22:43:28 +01:00
Sam Rose 182a1df960
Fix the bug, I think. 2024-03-05 17:35:04 +00:00
Adria Navarro db75c05942 Use scim-logs db 2024-03-05 18:25:23 +01:00
Adria Navarro 71c5d2645f Merge branch 'BUDI-8064/doc-writethrough' into BUDI-8046/scim-logger 2024-03-05 18:13:59 +01:00
Adria Navarro ebcb7718b8 Use bulk 2024-03-05 18:11:50 +01:00
Adria Navarro 4fe7e67dd5 Do not use lock 2024-03-05 18:11:50 +01:00
Adria Navarro 2b25f9f0cb Improve redlock non executed response 2024-03-05 18:11:50 +01:00
Adria Navarro 0649497ab5 Add comment 2024-03-05 18:11:50 +01:00
Adria Navarro 8d87850765 Remove defaults and init 2024-03-05 18:11:50 +01:00
Adria Navarro b94d28b7d6 Clean 2024-03-05 18:11:50 +01:00
Adria Navarro 420b0ffc03 Tests 2024-03-05 18:11:50 +01:00
Adria Navarro 936ddafee7 Use bull 2024-03-05 18:11:50 +01:00
Adria Navarro df325e21c3 Add doc-writethrough queue 2024-03-05 18:11:50 +01:00
Adria Navarro 2412d75cac Type inMemoryQueue 2024-03-05 18:11:49 +01:00
Adria Navarro 774ff42f0c Use overloads 2024-03-05 18:11:49 +01:00
Adria Navarro 82a6f9027e Namespace key in redis by db 2024-03-05 18:11:49 +01:00
Adria Navarro da012c0f08 Namespace key in redis by db 2024-03-05 18:11:49 +01:00
Adria Navarro e986d34b87 Fix types 2024-03-05 18:11:49 +01:00
Adria Navarro dc84eb4e80 Type caches 2024-03-05 18:11:49 +01:00
Adria Navarro eb9a163394 Making code more readable 2024-03-05 18:11:49 +01:00
Adria Navarro fd93eb79d5 Fixes and tests 2024-03-05 18:11:49 +01:00
Adria Navarro 04fb279623 Extra tests 2024-03-05 18:11:49 +01:00
Adria Navarro 223637999a Ensure keys are removed 2024-03-05 18:11:49 +01:00
Adria Navarro 41dde9722f Test concurrency 2024-03-05 18:11:49 +01:00
Adria Navarro 35536592e6 Add extra tests 2024-03-05 18:11:49 +01:00
Adria Navarro bfde028e9b Add persisting tests 2024-03-05 18:11:49 +01:00
Adria Navarro 74aae19a7e docWritethrough test 2024-03-05 18:11:49 +01:00
Adria Navarro 82132d539d DatabaseImpl.docExists test 2024-03-05 18:11:49 +01:00
Adria Navarro 10568cccff USe get for doc exists 2024-03-05 18:11:49 +01:00
Adria Navarro 9f42ea6bbf DocWritethrough 2024-03-05 18:11:49 +01:00
Adria Navarro 40cc383c01 Create docWriteThrough redis cache 2024-03-05 18:11:49 +01:00
Adria Navarro a093cfca99 Fix test 2024-03-05 18:11:12 +01:00
Adria Navarro de0527384a Support complex objects 2024-03-05 18:10:45 +01:00
Adria Navarro 82ff748fd9 Add complex object tests 2024-03-05 18:10:33 +01:00
Adria Navarro f2330144de Clean 2024-03-05 17:57:28 +01:00
Dean 136eeefc26 Merge remote-tracking branch 'origin/master' into feature/app-list-actions 2024-03-05 16:24:14 +00:00
Adria Navarro 8f9e8b60c3 Fix types 2024-03-05 17:19:26 +01:00
Adria Navarro 38d557ea1c
Merge branch 'master' into BUDI-8046/redis-bulk-store 2024-03-05 17:11:13 +01:00
Adria Navarro 3baf981d48 Add TTL tests 2024-03-05 17:08:08 +01:00
Adria Navarro 1b0a943e13 Atomic expires 2024-03-05 17:04:46 +01:00
Adria Navarro d9a5899b27 Bulk store test 2024-03-05 16:49:45 +01:00
Adria Navarro 49db47e1fd Add bulk store 2024-03-05 16:46:33 +01:00
Adria Navarro a4288a9dd3 Basic test 2024-03-05 16:41:49 +01:00
Adria Navarro 56870bed5b Typings 2024-03-05 16:30:45 +01:00
Adria Navarro a5d6d094e6 Update types 2024-03-05 16:28:31 +01:00
Sam Rose 4170213c0d
Merge branch 'master' of github.com:budibase/budibase into budi-7710-user-groups-do-not-fully-support-custom-roles-5 2024-03-05 14:47:56 +00:00
Sam Rose 488cfea1f4
Fix typing. 2024-03-05 14:40:29 +00:00
Adria Navarro e076c0e5f5 Use typed redis clients 2024-03-05 15:10:51 +01:00
Adria Navarro 5bd5ea06e7 Merge branch 'BUDI-8064/doc-writethrough' into BUDI-8046/scim-logger 2024-03-05 14:20:09 +01:00
Adria Navarro ca0f583399 Remove defaults and init 2024-03-05 14:19:05 +01:00
Adria Navarro 82d2116ce0 Merge branch 'BUDI-8064/doc-writethrough' into BUDI-8046/scim-logger 2024-03-05 13:59:33 +01:00
Adria Navarro caf142f1db Clean 2024-03-05 13:55:07 +01:00
Adria Navarro e648503e4f Tests 2024-03-05 13:50:58 +01:00
Sam Rose aa124524d4
Add a simpler test. 2024-03-05 10:05:05 +00:00
Sam Rose fced2f3696
Respond to PR feedback. 2024-03-05 09:23:48 +00:00
Sam Rose 9d8c18337d
Type role controller. 2024-03-04 16:42:41 +00:00
Adria Navarro 2d84bc5da2 Use bull 2024-03-04 16:34:05 +01:00
Adria Navarro 91468d2569 Add doc-writethrough queue 2024-03-04 16:18:01 +01:00
Dean 41b55c28b7 Merge remote-tracking branch 'origin/master' into feature/app-list-actions 2024-03-04 15:13:47 +00:00
Adria Navarro 62fe394139 Merge branch 'BUDI-8046/improve-test-queues' into BUDI-8064/doc-writethrough 2024-03-04 15:41:10 +01:00
Adria Navarro 824dd1c1fc Type inMemoryQueue 2024-03-04 15:38:45 +01:00
Dean 42893c1218 Merge remote-tracking branch 'origin/master' into feature/app-list-actions 2024-03-04 10:00:03 +00:00
mike12345567 ad0c1a94c9 Fixing some typing that was added. 2024-03-01 13:59:42 +00:00
Adria Navarro 4ff2b36553 Use overloads 2024-03-01 13:59:51 +01:00
Adria Navarro 2da5cb3ddb Namespace key in redis by db 2024-03-01 13:41:40 +01:00
Adria Navarro 93e462b8c7 Namespace key in redis by db 2024-03-01 13:38:48 +01:00
Adria Navarro 4e53cb5143 Flags 2024-03-01 13:09:42 +01:00
Adria Navarro 3a341338a1 Log requests 2024-03-01 13:07:27 +01:00
Adria Navarro 1c17121568 Fix types 2024-03-01 11:12:31 +01:00
Adria Navarro 8611fade5e
Merge branch 'master' into BUDI-8064/doc-writethrough 2024-03-01 11:08:49 +01:00
Adria Navarro ff7c784342 Type caches 2024-03-01 11:04:55 +01:00
Adria Navarro 2b7c988823 Making code more readable 2024-03-01 11:04:30 +01:00
Adria Navarro 66751728bb Fixes and tests 2024-03-01 10:53:18 +01:00
Sam Rose f876c5c6e4
Merge master. 2024-02-29 17:34:16 +00:00
mike12345567 50bbbb2e06 Updating datasource controller types, this branched out a little bit to removing as many anys as possible, and allowing RowValue to correctly be returned from the allDocs function of the DB. 2024-02-29 16:28:00 +00:00
Adria Navarro 6b8f67ed41 Extra tests 2024-02-29 17:01:16 +01:00
Adria Navarro 3068e58c31 Ensure keys are removed 2024-02-29 16:48:16 +01:00
Adria Navarro 720d5a4105 Test concurrency 2024-02-29 15:51:42 +01:00
Adria Navarro 3ec0052481 Add extra tests 2024-02-29 15:41:26 +01:00
Adria Navarro dc4d1fdbda Add persisting tests 2024-02-29 15:28:35 +01:00
Adria Navarro 40d7a0a741 docWritethrough test 2024-02-29 15:23:32 +01:00
Adria Navarro 3af2da3b7d DatabaseImpl.docExists test 2024-02-29 15:17:18 +01:00
mike12345567 7641e84430 Merge branch 'revert-13160-revert-13128-feature/sql-query-aliasing' of github.com:Budibase/budibase into labday/sqs 2024-02-29 13:14:37 +00:00
Adria Navarro 7d50a70d03 USe get for doc exists 2024-02-29 13:47:18 +01:00
Adria Navarro ff7c8d3b95 DocWritethrough 2024-02-29 13:44:52 +01:00
Sam Rose 6b306266b5
Only show stack traces if you ask for them. 2024-02-29 11:09:39 +00:00
Adria Navarro 76b9cbcc5f Create docWriteThrough redis cache 2024-02-29 11:22:27 +01:00
Sam Rose 8488ff4144
Print stack traces from inside request handler. 2024-02-29 09:19:14 +00:00
mike12345567 71ad1fde0f Fixing some build/type issues. 2024-02-28 17:16:36 +00:00
mike12345567 2d75e1e85c Merge branch 'feature/sql-query-aliasing' of github.com:Budibase/budibase into labday/sqs 2024-02-28 17:03:59 +00:00
Sam Rose 237634386c
More typing fixes. 2024-02-28 12:19:08 +00:00
Sam Rose 2e8eda47f1
Respond to PR feedback. 2024-02-26 17:38:33 +00:00
Sam Rose 8f61b69ee5
Merge branch 'master' of github.com:budibase/budibase into budi-7710-user-groups-do-not-fully-support-custom-roles 2024-02-26 17:28:51 +00:00
Sam Rose c15554547b
Respond to PR feedback. 2024-02-26 17:28:37 +00:00
mike12345567 90c06e633b Merging latest master. 2024-02-26 15:46:00 +00:00
Sam Rose b9600d8330
More progress on modernising application tests. 2024-02-26 11:57:56 +00:00
Dean 8556a869b9 Merge remote-tracking branch 'origin/master' into feature/app-list-actions 2024-02-26 11:01:01 +00:00
jvcalderon eafb8a6903 Merge branch 'master' into feature/synchronicity 2024-02-23 16:49:24 +01:00
jvcalderon 1ef5e54c29 Add BPM_EVENTS to Redis DB list 2024-02-23 16:26:23 +01:00
Dean 6877d0a016 Merge remote-tracking branch 'origin/master' into feature/app-list-actions 2024-02-22 15:00:34 +00:00
Dean 18f09f4e13 Duplicate app behaviour and test updates 2024-02-22 15:00:16 +00:00
Adria Navarro 8e77365c19 Allow mocking multiple features 2024-02-22 13:55:38 +01:00
Adria Navarro fb1bfbdae8 Clean code 2024-02-22 13:55:26 +01:00
Sam Rose ba66bc3232
Merge master. 2024-02-21 15:06:59 +00:00
Sam Rose 686697e890
Enforce using example.com as a domain for emails. 2024-02-21 11:30:22 +00:00
melohagan c12e5fd196
Fix array type missing from query schema selector (#12772)
* Tidy MongoDB aggregation pipeline view

* Remove unused code

* WIP

* Add bindings for bindings drawer

* Is not external table if it's a query

* Add QueryArrayFetch

* Bug fix

* JavaScript is the worst

* refactor

* Add array label to query schema

* Remove console log

* type fix

* Don't include Array in SchemaTypeOptions, but show label

* Fix bindings

* refactor

* Rename isObject to hasSchema

* WIP

* Typing WIP

* Type not Types

* Unused import

* type fix

* Handle json array subtype

* Support queryarray datasource type

* refactor

* yarn lock

* update account portal

---------

Co-authored-by: Sam Rose <hello@samwho.dev>
2024-02-19 09:13:03 +00:00
mike12345567 364579acc4 Fixing an issue when invalid version is supplied (not semver) shouldn't crash server. 2024-02-16 15:41:36 +00:00
Sam Rose e4b0330866
Simplify the typing of workerRequests.ts 2024-02-15 15:49:30 +00:00
Sam Rose dde8f77877
Type workerRequests.ts 2024-02-15 14:48:47 +00:00
Adria Navarro 4f42fd832d Lint 2024-02-09 16:47:58 +01:00
Adria Navarro 7ce9756d8c Revert jsRunner changes to vm 2024-02-09 16:36:43 +01:00
Sam Rose 77b3d479a0
Update dd-trace to 5.2.0. 2024-02-09 12:01:37 +00:00
Adria Navarro 573096e869
Merge branch 'master' into revert-12934-revert-12930-revert-12929-revert-12769-isolated-vm 2024-02-08 18:15:36 +01:00
Sam Rose 8e9db069e5
Remove all uses of test.com in the code, replace them with example.com and create a lint rule to prevent this in future. 2024-02-08 16:32:14 +00:00
Adria Navarro 3b8b60aa03 Use wrapper 2024-02-08 09:38:09 +01:00
Adria Navarro 218ba1d283 VM types 2024-02-08 09:38:09 +01:00
Adria Navarro d0d767ce5e
Merge branch 'master' into revert-12934-revert-12930-revert-12929-revert-12769-isolated-vm 2024-02-08 09:37:39 +01:00
mike12345567 7dc2c3551f Updating the listObjects functionality to correctly handle truncated responses, when not all objects can be returned at once we need to loop, but we weren't correctly picking up the token that should be passed. 2024-02-07 17:03:44 +00:00
Martin McKeaveney b17bd4d8fd merge 2024-02-02 17:00:09 -03:00
mike12345567 d87c7a1c4a Merge branch 'master' of github.com:Budibase/budibase into labday/sqs 2024-02-02 17:09:54 +00:00
Adria Navarro 8dfcc3aeb1
Revert "Revert "Migrate from `vm` to `isolated-vm`"" 2024-02-02 12:02:09 +01:00
Sam Rose 332a201ae7
Revert "Migrate from `vm` to `isolated-vm`" 2024-02-02 09:51:46 +00:00
Sam Rose ba002f9649
Clean up isolates when a request is finished. 2024-02-02 09:30:33 +00:00
Adria Navarro 45ea6a6fd6
Revert "Revert "Migrate from `vm` to `isolated-vm`"" 2024-02-01 16:47:41 +01:00
Adria Navarro 16749ec3dd
Revert "Migrate from `vm` to `isolated-vm`" 2024-02-01 16:30:34 +01:00
Adria Navarro 87501bf25e
Merge branch 'master' into isolated-vm 2024-02-01 13:15:47 +01:00
Adria Navarro 2ea70e1010
Merge pull request #12922 from Budibase/test-isolated-vm
Test isolated vm
2024-02-01 13:15:33 +01:00
Adria Navarro 3aff1004c8
Merge branch 'master' into isolated-vm 2024-02-01 12:42:48 +01:00
Adria Navarro 181d462cdf Update uuid 2024-02-01 11:15:21 +01:00
mike12345567 85f7d66a99 Handling very large exports/backup downloads. 2024-01-31 17:16:51 +00:00
Sam Rose 682c3c6c14
Merge branch 'master' into update-nano 2024-01-31 14:43:44 +00:00
Martin McKeaveney a23f76b8c8 update expiry time for attachments/images 2024-01-31 08:57:54 -03:00
Sam Rose f3e0dfd466
Update to @budibase/nano 10.1.5 2024-01-31 11:55:29 +00:00
Adria Navarro 623b4f5dab
Merge branch 'master' into isolated-vm 2024-01-31 10:47:08 +01:00
mike12345567 21a8d44ec7 Adding a 'SESSION_EXPIRY_SECONDS' environment variable which can be set on the services to configure how long before an idle user is logged out. 2024-01-25 16:22:39 +00:00
Adria Navarro bc7825dc93 Keep isolateRefs in context 2024-01-24 14:02:34 +01:00
Adria Navarro 09d8951fc3
Merge branch 'master' into isolated-vm 2024-01-18 14:21:13 +01:00
jvcalderon 64d3114c9f Refactor: usersUtils -> userUtils 2024-01-18 12:03:26 +01:00
jvcalderon 03f7fb37ed Calculate creators count when group role changes 2024-01-18 11:14:25 +01:00
mike12345567 b370e2a0e5 Updating dd-trace to 5.0.0 across all packages now that the memory leak is solved, this also removes the erronous message on startup describing DataDog startup problems. 2024-01-15 17:44:52 +00:00
Adria Navarro 73fee7dec7 Fix build 2024-01-15 12:04:35 +01:00
Sam Rose c508a435d6
Isolate and context re-use. 2024-01-12 15:08:08 +00:00
mike12345567 e69ba75849 Fixing typing for performCall. 2024-01-11 17:17:38 +00:00
mike12345567 8483a53178 This removes the need for constant DB HEAD requests to check if a database exists or not. Instead, it tries to make the request, and if it fails it will check if the reason for failure is the database not existing. If it doesn't exist it runs through the same old flow to confirm that it definitely doesn't exist, and if it doesn't then it will create it. 2024-01-11 16:44:15 +00:00
Martin McKeaveney a34ae9e2e2 merge with master 2024-01-08 16:44:32 +00:00
Sam Rose 7ca748a3be
Remove objects from setUser call. We can add them later if we need them. 2024-01-05 18:10:23 +00:00
Sam Rose 26fd6b55cd
Merge branch 'master' into attach-user-to-spans 2024-01-05 14:46:53 +00:00
Sam Rose 946a73f1b6
Set user ID correctly. 2024-01-05 14:40:16 +00:00
Michael Drury 9004428ef3
Merge branch 'master' into fix/password-length-admin-user-startup 2024-01-05 14:06:37 +00:00
mike12345567 b4b8e16f22 PR comments. 2024-01-05 13:58:31 +00:00
Sam Rose b3cd74e877
Remove user email from DataDog user info. 2024-01-05 13:49:01 +00:00
mike12345567 f2f16cfcee Adding an option to disable password validation when creating an admin user - this means that the environment variables used for BB_ADMIN creation can have any length of password (not breaking change). 2024-01-05 13:17:10 +00:00
Sam Rose 69bfd01bf0
Attach authenticated user to DataDog spans. 2024-01-05 11:37:10 +00:00
mike12345567 bbeaed48a7 Removing the use of the addQueries quota functionality from row API - we don't make use of these statistics anymore (and haven't in a while) counting them is pointless and could be causing some loss in performance in these heavily used endpoints. 2024-01-04 17:38:54 +00:00
Sam Rose 68468fadb3
Add an extra JS execution time limit check to prevent creating unnecesary VM context. 2024-01-04 10:10:00 +00:00
Adria Navarro 7413fd96a8
Merge branch 'master' into chore/node-20 2024-01-03 14:15:21 +01:00
Michael Drury e0a550e098
Merge branch 'master' into fix/multi-tenancy-dev-access 2024-01-03 12:57:17 +00:00