Commit Graph

3640 Commits

Author SHA1 Message Date
Martin McKeaveney 537c92b371 tidy 2024-07-22 21:23:27 +01:00
Martin McKeaveney 88ea3fdc07 unit tests and logic added to prevent coalesce being used by oracle 2024-07-22 21:00:20 +01:00
Sam Rose f3c73fe4a8
Support primitives in feature flags, make flag types flow, remove some obsolete feature flag systems. 2024-07-22 17:43:53 +01:00
Sam Rose c64d76eb84
Respond to PR comments. 2024-07-19 15:07:58 +01:00
Sam Rose 669692e722
Make switch exhaustive. 2024-07-18 11:41:37 +01:00
Sam Rose 481bf9a8b8
Fix generic-sql.spec.ts 2024-07-18 11:40:44 +01:00
Sam Rose df765c1dc7
Merge branch 'master' of github.com:budibase/budibase into support-spaces-as-last-char-in-column-sqs 2024-07-18 11:12:25 +01:00
Sam Rose 0b2a5162a4
Fix the problem, and the tests. 2024-07-18 11:00:10 +01:00
Sam Rose aea9cda8f5
wip 2024-07-17 15:45:35 +01:00
Adria Navarro 1c69cfaeda Fix initialisation 2024-07-12 12:32:36 +02:00
Adria Navarro 50c8449f4b Add extra test 2024-07-12 12:17:37 +02:00
Sam Rose e70e7eeb1d
Merge branch 'master' into fix-flake 2024-07-11 15:08:59 +01:00
Sam Rose d530909c3b
Fix a flake in the redis.spec.ts file. 2024-07-11 13:58:28 +01:00
mike12345567 8d308b591a Enabled SQS in self-host/development (single tenancy) as long as the SQS_ENABLED flag has been set, tenants don't need to be specified as there are no tenants. 2024-07-11 13:33:56 +01:00
mike12345567 73881e9895 Only default the CouchDB SQL URL if we are in dev, otherwise attempt to work out what it is based on the main CouchDB URL (as this should work in all default production environments). 2024-07-11 11:11:28 +01:00
Sam Rose 56c6742a0e
Create a way to enable SQS for all tenants. 2024-07-10 15:22:18 +01:00
Sam Rose 551d413900
Merge branch 'master' into sqs-per-tenant 2024-07-10 11:39:31 +01:00
Sam Rose 5356cfdce5
Fix uses of SearchFilterOperator. 2024-07-10 11:08:11 +01:00
Sam Rose 2fa14ab35f
Merge branch 'master' into sqs-per-tenant 2024-07-09 17:30:33 +01:00
Adria Navarro 38f7b88735 Type filters 2024-07-09 14:01:44 +02:00
Adria Navarro 9c6347f7fd Move constants to types 2024-07-09 12:24:59 +02:00
Adria Navarro 5cba1058d4
Merge branch 'master' into budi-7065-csv-export-fails-for-table-with-composite-primary-key 2024-07-09 10:54:18 +02:00
Adria Navarro 5387717183 Fix 2024-07-09 10:43:53 +02:00
Adria Navarro 5be8882122 Handle composite keys on exports 2024-07-09 10:43:45 +02:00
Sam Rose 014ff81841
Remove accidental imports of the aws-sdk SQS (fml) 2024-07-08 14:25:05 +01:00
Sam Rose 69d54b523d
Finish implementation, fix tests. 2024-07-08 14:21:07 +01:00
Sam Rose 1851e11bc0
wip 2024-07-08 13:28:48 +01:00
Adria Navarro 85827bbf93 Refactor breakRowIdField 2024-07-08 13:27:30 +02:00
mike12345567 e4375c2196 Fixing a build issue uncovered by tests. 2024-07-05 15:27:54 +01:00
mike12345567 62ca71f00e Merge branch 'master' of github.com:Budibase/budibase into fix/invalid-time-value 2024-07-05 15:02:36 +01:00
mike12345567 77abe6da83 Handling invalid time values when ISO strings are input as filter options. 2024-07-04 18:29:08 +01:00
Sam Rose a1d07e0ef0
Merge branch 'master' of github.com:budibase/budibase into budi-8417-check-error-responses-in-middleware-for-environment 2024-07-04 13:46:48 +01:00
Adria Navarro ddcf5598b4
Merge branch 'master' into chore/fix-npm-vulnerabilities 2024-07-04 12:30:40 +02:00
Sam Rose 257ee8fb70
Fix tests actually. 2024-07-04 10:46:09 +01:00
Adria Navarro 7b51799e8d depcheck clean 2024-07-04 11:22:54 +02:00
Adria Navarro 8da83dac58 Merge branch 'master' into chore/fix-npm-vulnerabilities 2024-07-04 11:08:51 +02:00
Sam Rose 16e293a9ff
Fix tests. 2024-07-04 09:55:36 +01:00
Michael Drury 4933658b98
Merge branch 'master' into fix/couchdb-integration 2024-07-03 17:36:16 +01:00
Sam Rose 4f01cb50cd
Merge branch 'master' of github.com:budibase/budibase into budi-8417-check-error-responses-in-middleware-for-environment 2024-07-03 17:16:44 +01:00
Sam Rose 75360bdb9b
Merge branch 'master' into merge-env-vars 2024-07-03 17:06:14 +01:00
Sam Rose bab3c07727
Add a couple more secrets. 2024-07-03 16:33:32 +01:00
Sam Rose d9b94c1dcf
Don't detect empty strings. 2024-07-03 16:31:42 +01:00
Sam Rose 253110ac6f
Detect secrets in error messages. 2024-07-03 16:31:42 +01:00
Sam Rose cd98882127
Move secrets into backend-core. 2024-07-03 16:30:23 +01:00
Adria Navarro e74847585e
Merge branch 'master' into BUDI-8416/allow-updating-email-via-scim 2024-07-03 15:59:47 +02:00
Adria Navarro 8fd2cce093 Remove platform user on remove 2024-07-03 15:52:48 +02:00
mike12345567 cb41e4d5a1 Fix for CouchDB integration, the typing in it was very poor and wise hiding issues, I've updated how this is implemented so that static typing can catch issues with it. 2024-07-03 12:50:33 +01:00
mike12345567 cb50cca40b Stripping credentials from Couch URLs + test cases. 2024-07-03 11:25:21 +01:00
Adria Navarro 19e4e8fdb4 Allow updating email 2024-07-03 12:18:06 +02:00
Adria Navarro 3ed9c9a4a5 Test 2024-07-03 12:15:56 +02:00
Adria Navarro 88e054c366 Basic update test 2024-07-03 12:06:37 +02:00
Adria Navarro 4ddd450a89 More tests 2024-07-03 11:59:12 +02:00
Adria Navarro 8372632579 Add extra tests 2024-07-03 11:14:30 +02:00
Adria Navarro 6510a47c06 Add basic userDB.save test 2024-07-03 11:10:51 +02:00
Adria Navarro ae4cfd7b47 Fix pouchdb-replication-stream vulnerability 2024-07-02 15:23:47 +02:00
Michael Drury c1f7561839
Merge branch 'master' into fix/sqs-internal-columns 2024-06-27 18:19:03 +01:00
mike12345567 3394e974d3 Getting casting working again, have to pass down prefix for checking against fields. 2024-06-27 18:18:41 +01:00
Michael Drury cfeab17ed8
Revert "Revert "Disallow prohibited columns"" 2024-06-26 12:40:16 +01:00
Michael Drury 9c92b8eff9
Merge branch 'master' into sustaining/typescript-5.5.2 2024-06-26 11:03:01 +01:00
Sam Rose 314d62bea0
Revert "Disallow prohibited columns" 2024-06-25 14:29:56 +01:00
melohagan 4ac9b657e5
Remove deprecated properties (#13958)
* Remove deprecated properties

* Fix backend-core test

* Don't run account-portal tests

* Update account-portal ref

* Run account portal unit tests

* Revert "Run account portal unit tests"

This reverts commit b509bf31a4.

* Revert "Update account-portal ref"

This reverts commit 186391fbb6.
2024-06-25 08:26:52 +01:00
mike12345567 a7b995e2ad Merge branch 'master' of github.com:Budibase/budibase into sustaining/typescript-5.5.2 2024-06-24 18:05:32 +01:00
Sam Rose bb91849976
Merge branch 'master' into budi-8390-receiving-a-column-has-been-specified-more-than-once-in-the 2024-06-24 17:32:26 +01:00
Sam Rose 58d8f2bb64
Respond to PR feedback. 2024-06-24 17:30:10 +01:00
Sam Rose da1eb6f6ac
Fix primary key appearing twice in the SQL ORDER BY clause. 2024-06-24 17:09:27 +01:00
mike12345567 ebf8f54bb7 Upgrading to Typescript 5.5.2 2024-06-24 16:41:28 +01:00
Sam Rose 01b2f69b74
Merge branch 'master' of github.com:budibase/budibase into budi-8220-support-updating-existing-rows-for-external-databases-using-2 2024-06-24 15:51:03 +01:00
Michael Drury 0d946e7be3
Merge branch 'master' into fix/relationship-filtering-table-or-column 2024-06-24 14:35:30 +01:00
Peter Clement aefe46b253
Adds _id and _rev back to internal datasource filter options (#13977)
* Adds _id and _rev back to internal datasource filter options

* add bb default datasource const into shared-core

* re-export var from shared-core
2024-06-24 14:31:27 +01:00
mike12345567 1402716f5c Some type updates. 2024-06-24 13:10:30 +01:00
Sam Rose f4378bd561
Merge branch 'master' of github.com:budibase/budibase into budi-8220-support-updating-existing-rows-for-external-databases-using-2 2024-06-21 13:08:39 +01:00
mike12345567 def3b0260e Disallowing prohibited columns consistently, no matter the case, and backend validation for this as well. 2024-06-20 18:48:22 +01:00
mike12345567 67c00c9e4c Addressing PR comments. 2024-06-19 18:46:48 +01:00
Sam Rose 602faf1c67
Add test for composite primary keys for external datasource imports. 2024-06-19 17:52:48 +01:00
mike12345567 2aa911b217 re-jigging things to get counting working properly again. 2024-06-19 17:10:15 +01:00
mike12345567 2d74927177 updating how counting disables sorting. 2024-06-19 15:08:12 +01:00
mike12345567 1056efdbf6 Changing how counting occurs in SQL layer. 2024-06-19 14:56:06 +01:00
Michael Drury 223df424fc
Merge branch 'master' into feature/count-rows-to-return 2024-06-19 14:20:31 +01:00
mike12345567 bdbb4c0a66 Commenting on a bug in knex 2024-06-19 13:59:03 +01:00
mike12345567 0de94d3535 Addressing some PR comments. 2024-06-19 11:52:50 +01:00
mike12345567 e0d8a66fd8 Adding test cases for counting as well as some small fixes. 2024-06-18 18:44:17 +01:00
Sam Rose e288fc8795
Disable upserting for MSSQL in bulkImport for now. 2024-06-18 18:02:20 +01:00
Sam Rose 5ac8a7d514
bulkImport upsert working everywhere excpet mssql 2024-06-18 17:43:25 +01:00
mike12345567 5c45370722 Fixing an issue with the sort order not being deterministic consistently. 2024-06-18 13:39:10 +01:00
mike12345567 278242fb55 Moving limits and offsets back into pre-query. 2024-06-18 12:47:59 +01:00
mike12345567 03b82c0923 Merge branch 'master' of github.com:Budibase/budibase into feature/count-rows-to-return 2024-06-17 13:41:40 +01:00
Sam Rose 6516a01657
Merge branch 'master' into uncomment-search-test-todos 2024-06-17 13:34:53 +01:00
melohagan 9ecf82e0f6
Add endpoint to GET tenant info from tenant global DB. (#13923)
* Add GET tenant info endpoint

* Add GET global tenant to endpoint list.

* Use correct not found code

* Fix unit test

* Code review comments

* Ignore account portal check types

* remove account-portal-ui from type check ignore

* Remove ignore account-portal-ui from type check
2024-06-17 09:22:44 +00:00
mike12345567 1b36d8af51 Getting counting flow working correctly for external datasources. 2024-06-14 19:00:59 +01:00
mike12345567 e402f9c571 Merge branch 'uncomment-search-test-todos' of github.com:Budibase/budibase into feature/count-rows-to-return 2024-06-14 18:16:47 +01:00
mike12345567 77556820bf Bit more work towards row counting, as well as moving external SQL to use row + 1 for working out pagination. 2024-06-14 18:12:36 +01:00
Sam Rose a54048e308
Solve all remaining TODOs in search.spec.ts. 2024-06-14 14:57:31 +01:00
mike12345567 2c6262844b Some work to limiting, changing how limiting works for pagination so that filtering on relationships doesn't cause problems. 2024-06-14 14:35:35 +01:00
Sam Rose c27e9c266c
Replace SortDirection with SortOrder, SortDirection is being removed. 2024-06-14 09:45:30 +01:00
Sam Rose a140a002e2
Merge branch 'master' of github.com:budibase/budibase into fix/aws-session-token-s3-ver2 2024-06-11 18:04:40 +01:00
Michael Drury 869c7e546a
Merge branch 'master' into disable-app-migration-cache 2024-06-11 13:17:16 +01:00
melohagan 115737c46c
Create Tenant endpoint + tenant_info doc (#13902)
* Create Tenant endpoint + tenant_info doc

* Don't catch on tenant_info put

* PR comments

* unit test
2024-06-11 12:16:15 +01:00
mike12345567 334334bbcd Adding a check for QA. 2024-06-11 12:13:18 +01:00
mike12345567 75c3b842ad Fixing issue with in memory queue. 2024-06-10 23:34:08 +01:00
mike12345567 1bd5a41712 Setting debug properly 2024-06-10 15:29:32 +01:00
mike12345567 fe789e7462 Adding test containers debug. 2024-06-10 15:17:01 +01:00
Hector Valcarcel 65c2039d0c Chore: Allow using an AWS_SESSION_TOKEN for object storage with AWS S3 2024-06-10 11:48:54 +01:00
mike12345567 ee391b30d6 Fixing an issue with multi-tenant app migration failures - the app migration context wasn't getting the tenant ID as needed. Also added some better logging as well as cleaning up the queues properly to remove these dud events. 2024-06-07 22:50:06 +01:00
Michael Drury b76d535202
Merge branch 'master' into BUDI-7656/add-migration 2024-06-07 10:24:50 +01:00
mike12345567 ab6614a683 Capturing any 500s which occur from SQS cleanup - it appears to 500 if there is nothing to do. 2024-06-06 17:38:52 +01:00
Michael Drury ea1d3000c2
Merge branch 'master' into BUDI-7656/add-migration 2024-06-06 16:49:48 +01:00
mike12345567 70aa43680d Addressing PR comments. 2024-06-06 16:49:03 +01:00
mike12345567 2c86ea14fd Changing how disk cleanup is performed. 2024-06-06 15:07:06 +01:00
Sam Rose ac53a97176
WIP 2024-06-06 14:55:07 +01:00
mike12345567 4f6076d2ba Merge branch 'master' of github.com:Budibase/budibase into BUDI-7656/add-migration 2024-06-05 16:29:45 +01:00
mike12345567 cf6f86cb2f Updating migrations to correctly cover all the required elements. 2024-06-04 17:24:11 +01:00
mike12345567 e3b1655ad0 Merge branch 'master' of github.com:Budibase/budibase into BUDI-7656/add-migration 2024-06-04 12:56:59 +01:00
Adria Navarro a562655361
Merge branch 'master' into feat/readonly-columns 2024-06-04 12:04:49 +02:00
Adria Navarro 2d953f19cc Clean validation message 2024-06-04 11:11:50 +02:00
mike12345567 155de99b68 Streaming to disk before passing onto S3. 2024-06-03 11:46:20 +01:00
mike12345567 75501c2251 Updating object store stream upload to make sure the stream has finished being processed before trying to upload to AWS (and only uploading a partial stream). 2024-05-31 17:57:31 +01:00
mike12345567 f3d1a8aabc Merge branch 'master' of github.com:Budibase/budibase into BUDI-7656/add-migration 2024-05-29 17:20:29 +01:00
Michael Drury aa232db669
Merge branch 'master' into fix/template-app-rows 2024-05-29 15:06:09 +01:00
Adria Navarro f5f471a4aa Merge branch 'master' into BUDI-8284/readonly-settings-columns-in-views 2024-05-29 15:41:20 +02:00
mike12345567 6b7a4f07cb Quick fix for ambiguous column names _id. 2024-05-29 11:56:52 +01:00
mike12345567 83fa6f952a re-jigging, making this part of filters rather than meta. 2024-05-28 18:34:08 +01:00
mike12345567 cb99007a42 Fix for issue with templated apps - some template apps are *very* old and therefore don't necessarily abide by the rules that are generally followed for new apps, this is something we should resolve, but for now adding an option that allows searching for only a specific document type. 2024-05-28 18:30:06 +01:00
Adria Navarro 422b1e2753 Fix tests 2024-05-27 14:31:45 +02:00
mike12345567 3b310b188e Fixing issue with column quoting. 2024-05-24 17:52:40 +01:00
mike12345567 b18ca2670a Merge branch 'master' of github.com:Budibase/budibase into feature/audit-log-sqs 2024-05-24 12:22:20 +01:00
Michael Drury a3d2871330
Merge branch 'master' into feature/audit-log-sqs 2024-05-23 16:52:50 +01:00
mike12345567 66686fd970 Making sure all JSON column types are correctly handled. 2024-05-23 15:00:13 +01:00
mike12345567 0c28d05d40 Some work to correctly handle JSON columns from SQS as well. 2024-05-23 14:57:38 +01:00
mike12345567 99536c2d0b PR comments. 2024-05-23 14:24:49 +01:00
Adria Navarro e169454490 Move utils to backend-core 2024-05-23 14:59:51 +02:00
mike12345567 ff8684d9b4 Merge branch 'master' of github.com:Budibase/budibase into feature/audit-log-sqs 2024-05-22 15:23:24 +01:00
mike12345567 a2200a38fb Merge branch 'master' of github.com:Budibase/budibase into feature/audit-log-sqs 2024-05-20 14:09:57 +01:00
mike12345567 d1b6a542fa Getting audit log searching with sqs working - a bit more work than expected but fixed quite a few SQS bugs along the way. 2024-05-17 20:26:34 +01:00
mike12345567 d7f3109a22 Making sqlite design doc generation more accessible. 2024-05-17 12:35:31 +01:00
Adria Navarro 19cb7b83e0
Merge branch 'master' into BUDI-8248/reduce-googleapi-calls 2024-05-17 12:28:28 +02:00
Adria Navarro 7807b734bb Persist googlesheet refs in context 2024-05-17 12:07:28 +02:00
mike12345567 0efa1f06ab Moving some stuff around to make way for other services using the sql layers. 2024-05-16 17:33:47 +01:00
mike12345567 647a8c2a74 Final fix for remove function. 2024-05-16 13:52:38 +01:00
mike12345567 06d6d84b55 Addressing PR comments. 2024-05-16 13:27:54 +01:00
mike12345567 aa51db20ee Changing how cleanup works - the cleanup is now part of the DB deletion, making sure it cannot be missed. 2024-05-14 15:41:48 +01:00
mike12345567 673211dfbb Merge branch 'master' of github.com:Budibase/budibase into feature/sqs-table-cleanup 2024-05-14 14:38:00 +01:00
mike12345567 39147b80b6 PR comments. 2024-05-14 11:23:50 +01:00
mike12345567 5b5e7e47a2 Fixing some issues with updating rows with attachments, there were some UI bugs that had been raised (undefined being displayed) and the uploading from a URL was not working as expected, due to the stream not matching the expected format of the S3 SDK - this has been fixed by converting the stream to a true readable stream with . 2024-05-13 18:11:40 +01:00
mike12345567 a8e0443cf7 Merge branch 'master' of github.com:Budibase/budibase into feature/sqs-table-cleanup 2024-05-13 14:01:14 +01:00
mike12345567 f25703b88e Merge branch 'master' of github.com:Budibase/budibase into feature/sqs-table-cleanup 2024-05-13 13:54:00 +01:00
Adria Navarro 2c68c3ffb1 Merge branch 'master' into chore/remove-unused-packages 2024-05-13 14:38:46 +02:00
Adria Navarro 4bec2a3a48
Merge branch 'master' into feat/budi-8123-single-user 2024-05-13 10:53:49 +02:00
mike12345567 10608f9bb7 Final final fix. 2024-05-10 11:59:11 +01:00
mike12345567 de2d0e6b89 Adding error field. 2024-05-10 11:51:57 +01:00
mike12345567 1aa89c61b6 One small change to keep 404 functionality on destroy DB. 2024-05-10 11:32:57 +01:00