Commit Graph

7317 Commits

Author SHA1 Message Date
Dean a85fd8c393 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-27 20:18:02 +01:00
mike12345567 ade514b96f Merge branch 'fix/sqs-internal-columns' of github.com:Budibase/budibase into fix/sqs-internal-columns 2024-06-27 18:30:40 +01:00
mike12345567 6b4852bf3e Updating migration test case. 2024-06-27 18:30:22 +01:00
Michael Drury c1f7561839
Merge branch 'master' into fix/sqs-internal-columns 2024-06-27 18:19:03 +01:00
mike12345567 3394e974d3 Getting casting working again, have to pass down prefix for checking against fields. 2024-06-27 18:18:41 +01:00
mike12345567 b0e6d3e72c Correctly handling overlapping column names. 2024-06-27 17:55:09 +01:00
Dean 23b0f3dfe4 Added missing rowProcessor mapping config for BB_Reference, BB_Reference_Single and Attachment_Single 2024-06-27 14:25:32 +01:00
Michael Drury f092bf46cf
Merge branch 'master' into revert-14020-revert-13993-fix/disallow-prohibited-columns 2024-06-27 13:30:00 +01:00
Dean b6d7715831 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-27 12:49:26 +01:00
Peter Clement cf11009e24
Update OpenAI dependency to fix 400 errors (#14033)
* Update OpenAI dependency to fix 400 errors

* add some optional chaining for safety

* add caret

* fix tests
2024-06-27 12:17:48 +01:00
Dean 2243314314 Defaulting to null in the updateRow step causes issues in row mapping. Switching back to empty string 2024-06-27 09:10:28 +01:00
mike12345567 1b9211ee6c Initial implementation of solving SQS issue with case insensitivity. 2024-06-26 16:40:20 +01:00
Dean d10ef7b795 Alter logic for field config checks causing test issues 2024-06-26 15:34:00 +01:00
Dean ca44220bd3 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-26 14:30:08 +01:00
Dean 21c335caab Lint and tidying up 2024-06-26 14:29:42 +01:00
mike12345567 41f045d8a6 Allow constant internal columns to be duplicated based on being case sensitive. 2024-06-26 13:36:20 +01:00
Michael Drury cfeab17ed8
Revert "Revert "Disallow prohibited columns"" 2024-06-26 12:40:16 +01:00
Dean 9adae8a9fd If a relationship value is set to null, the row processor will set it to an empty to array an ensure its properly cleared. Previously the empty value would be an empty string, which was then set to an empty array in the rowProcessor 2024-06-26 11:12:25 +01:00
Dean ed0f60d5d7 Added an app trigger update parser that will clean the testdata for the automation on save. Old values were not cleaned out. Added some padding to the PropField labels. General fixes 2024-06-26 11:10:15 +01:00
Michael Drury 9c92b8eff9
Merge branch 'master' into sustaining/typescript-5.5.2 2024-06-26 11:03:01 +01:00
Dean 1db1b638e6 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-26 09:08:11 +01:00
Sam Rose 314d62bea0
Revert "Disallow prohibited columns" 2024-06-25 14:29:56 +01:00
Dean 5ff8b03378 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-25 12:05:31 +01:00
Dean e9985adcd4 Updates and refactoring of Automation flows 2024-06-25 11:52:11 +01:00
Peter Clement 1fa18ccfce
Some automation fixes and refactoring (#13870)
* fix issue where booleans were being parsed to null

* refactor looping out of automations

* clean out execute function in orchestrator of looping code

* re-add accidentally deleted file

* remove spec file

* remove log

* move code back into main automation thread

* account portal update

---------

Co-authored-by: Adria Navarro <adria@budibase.com>
Co-authored-by: Michael Drury <me@michaeldrury.co.uk>
Co-authored-by: Sam Rose <hello@samwho.dev>
2024-06-25 11:17:14 +01:00
Peter Clement f3d466f255
fix issue where schema wasn't updating types when a query was run (#14004)
* fix issue where schema wasn't updating types when a query was run

* add tests for schema matching
2024-06-25 08:51:35 +01:00
mike12345567 a7b995e2ad Merge branch 'master' of github.com:Budibase/budibase into sustaining/typescript-5.5.2 2024-06-24 18:05:32 +01:00
Sam Rose bb91849976
Merge branch 'master' into budi-8390-receiving-a-column-has-been-specified-more-than-once-in-the 2024-06-24 17:32:26 +01:00
Sam Rose da1eb6f6ac
Fix primary key appearing twice in the SQL ORDER BY clause. 2024-06-24 17:09:27 +01:00
mike12345567 ebf8f54bb7 Upgrading to Typescript 5.5.2 2024-06-24 16:41:28 +01:00
Sam Rose 01b2f69b74
Merge branch 'master' of github.com:budibase/budibase into budi-8220-support-updating-existing-rows-for-external-databases-using-2 2024-06-24 15:51:03 +01:00
Sam Rose 2458259093
Respond to PR feedback. 2024-06-24 15:50:09 +01:00
mike12345567 b597bd3dbe Fixing an issue detected by tests. 2024-06-24 13:30:18 +01:00
mike12345567 75cee3c4fd Quick type improvement. 2024-06-24 13:28:13 +01:00
mike12345567 1402716f5c Some type updates. 2024-06-24 13:10:30 +01:00
mike12345567 965725d022 First PR comments. 2024-06-24 12:43:26 +01:00
mike12345567 05ea231d20 Adding back missing comments. 2024-06-24 11:53:02 +01:00
mike12345567 fcf67f7297 Fixing an issue raised by Poirazis around empty relationships coming back as related to themselves. 2024-06-21 19:29:30 +01:00
mike12345567 337584f5b2 Updating the regex to correctly find within the filter keys. 2024-06-21 17:51:02 +01:00
mike12345567 28d0d627ce Getting functions in place which make it easy to update pats of a filter list by their keys - getting this to work for SQS and external. 2024-06-21 17:00:12 +01:00
mike12345567 6812c21076 Updating test cases. 2024-06-21 16:58:40 +01:00
mike12345567 e5c40c7ecd Moving some stuff around inside ExternalRequests to make it easier to access parts of the full context. 2024-06-21 16:58:27 +01:00
Sam Rose d5481312d0
Fix lint, add new update test. 2024-06-21 15:31:34 +01:00
Sam Rose c9fecbaa77
Fix updating rows in external tables where the primary key is an autocolumn. 2024-06-21 15:12:02 +01:00
Sam Rose f4378bd561
Merge branch 'master' of github.com:budibase/budibase into budi-8220-support-updating-existing-rows-for-external-databases-using-2 2024-06-21 13:08:39 +01:00
mike12345567 b4910043c6 Addressing PR comments. 2024-06-21 11:27:47 +01:00
mike12345567 fead1f436a test case is only for internal. 2024-06-20 18:53:01 +01:00
mike12345567 ae68c561f4 Test case. 2024-06-20 18:51:04 +01:00
mike12345567 def3b0260e Disallowing prohibited columns consistently, no matter the case, and backend validation for this as well. 2024-06-20 18:48:22 +01:00
mike12345567 86bae92ada Refactoring search test to make it easier to find promises which aren't handled. 2024-06-20 17:13:42 +01:00
mike12345567 df56371ab6 Reverting change to promises. 2024-06-20 16:36:18 +01:00
mike12345567 295961edb1 Attempting without promise.all in external. 2024-06-20 16:18:32 +01:00
mike12345567 66ef0cb79a Adding back SQS - wasn't causing a problem. 2024-06-20 16:05:03 +01:00
mike12345567 b6b05e08b1 Removing SQS from view test to check. 2024-06-20 15:52:56 +01:00
mike12345567 3345364675 Updating test case - not exactly sure what it was testing before, but now it definitely confirms paginated results are stable. 2024-06-20 14:36:08 +01:00
mike12345567 580e36f301 Updating test case. 2024-06-20 10:46:52 +01:00
Dean d50a8e0746 Merge remote-tracking branch 'origin/master' into feature/automation-row-ux-update 2024-06-20 10:43:33 +01:00
mike12345567 86d9de5a2d Fixing view test cases, adding SQS to it and correcting the default sort order. 2024-06-19 18:57:37 +01:00
mike12345567 67c00c9e4c Addressing PR comments. 2024-06-19 18:46:48 +01:00
Sam Rose 602faf1c67
Add test for composite primary keys for external datasource imports. 2024-06-19 17:52:48 +01:00
mike12345567 58ec7a50b0 Implementing row counting for in-memory, also updating the in memory search function. 2024-06-19 17:51:11 +01:00
mike12345567 d121633d8e Updating queries to be a bit more flexible to updates in the SQL layers. 2024-06-19 17:29:33 +01:00
mike12345567 a97b24658f Fixing test case, it didn't provide a primary field. 2024-06-19 15:08:22 +01:00
mike12345567 aab100b130 Changing how counting is processed. 2024-06-19 14:28:22 +01:00
Michael Drury 223df424fc
Merge branch 'master' into feature/count-rows-to-return 2024-06-19 14:20:31 +01:00
mike12345567 bc80841554 Promise.all for both counts (SQS and SQL). 2024-06-19 13:39:00 +01:00
mike12345567 0e5de7f16d Test updates. 2024-06-19 13:36:34 +01:00
mike12345567 abfab054d7 Addressing comment about datasource being optional. 2024-06-19 12:03:20 +01:00
mike12345567 c973562398 Merge branch 'feature/count-rows-to-return' of github.com:Budibase/budibase into feature/count-rows-to-return 2024-06-19 11:53:11 +01:00
mike12345567 0de94d3535 Addressing some PR comments. 2024-06-19 11:52:50 +01:00
Sam Rose 80642100fa
Merge branch 'master' into feature/count-rows-to-return 2024-06-19 09:05:56 +01:00
mike12345567 cf75a8a1f3 Updating function name. 2024-06-18 18:49:59 +01:00
mike12345567 e0d8a66fd8 Adding test cases for counting as well as some small fixes. 2024-06-18 18:44:17 +01:00
Sam Rose e288fc8795
Disable upserting for MSSQL in bulkImport for now. 2024-06-18 18:02:20 +01:00
Sam Rose 5ac8a7d514
bulkImport upsert working everywhere excpet mssql 2024-06-18 17:43:25 +01:00
Sam Rose 335c174643
Create a failing test. 2024-06-18 16:39:35 +01:00
mike12345567 0caff1a404 Fixing an issue with sorting in SQS. 2024-06-18 13:53:51 +01:00
Peter Clement 2b96cbcad7
Expose old row binding in automations (#13931)
* expose old row through the emitter

* accidentally added oldRow to step

* fix row fetch in external datasources

* add test for new / old row comparison

* add testing for old row update event

* allow function overloading in test files

* update tests per comments

* handle event race condition

* update test data modal to account for old row output

* switch icon positioning
2024-06-18 13:45:58 +01:00
mike12345567 bda83205ee Making sure to measure whether we have paged forward in the query based on raw results. 2024-06-18 12:48:36 +01:00
mike12345567 73013332ae Adding countRows to search validators. 2024-06-18 11:16:21 +01:00
mike12345567 75bf3c78e6 Merge branch 'master' of github.com:Budibase/budibase into feature/count-rows-to-return 2024-06-18 11:07:34 +01:00
Sam Rose 75e79156f7
Merge branch 'master' into budi-8238-rewrite-postgresspects-to-use-knex 2024-06-17 16:45:33 +01:00
Sam Rose 903c3cf84d
Fix flake in generic-sql.spec.ts. 2024-06-17 16:17:00 +01:00
Sam Rose c107ab937d
Remove rawQuery entirely. 2024-06-17 16:15:51 +01:00
Sam Rose 16cacb3de7
Convert mysql.spec.ts away from rawQuery. 2024-06-17 15:48:58 +01:00
Sam Rose 84118f9e8c
Remove makeRequest from mysql.spec.ts 2024-06-17 14:39:45 +01:00
mike12345567 c34c219e8f Tidying up one of the weirder things knex can do. 2024-06-14 19:10:52 +01:00
mike12345567 654a417d66 Type checking. 2024-06-14 19:07:50 +01:00
mike12345567 cd1e7c0bad Small re-jig make things easier to read. 2024-06-14 19:04:02 +01:00
mike12345567 f3ca1d0b1e Adding countRows parameter to external API for counting. 2024-06-14 19:01:20 +01:00
mike12345567 1b36d8af51 Getting counting flow working correctly for external datasources. 2024-06-14 19:00:59 +01:00
mike12345567 908b77fd9b Fixing some issues with using offsets. 2024-06-14 18:27:23 +01:00
mike12345567 e402f9c571 Merge branch 'uncomment-search-test-todos' of github.com:Budibase/budibase into feature/count-rows-to-return 2024-06-14 18:16:47 +01:00
mike12345567 77556820bf Bit more work towards row counting, as well as moving external SQL to use row + 1 for working out pagination. 2024-06-14 18:12:36 +01:00
Sam Rose c2320e4f5b
Convert postgres.spec.ts to use Knex instead of rawQuery. 2024-06-14 17:20:08 +01:00
Sam Rose a54048e308
Solve all remaining TODOs in search.spec.ts. 2024-06-14 14:57:31 +01:00
mike12345567 7491021ca0 Base implementation of counting (plumbing). 2024-06-14 14:35:14 +01:00
Sam Rose 520f900708
Merge branch 'master' of github.com:budibase/budibase into budi-8349-in-memory-search-parity-testing 2024-06-14 13:59:06 +01:00
Sam Rose f3c82cbf38
Ensure that sort order is always lowercased. 2024-06-14 11:00:34 +01:00
Sam Rose c27e9c266c
Replace SortDirection with SortOrder, SortDirection is being removed. 2024-06-14 09:45:30 +01:00
mike12345567 82af6deab3 Making sure we return the correct thing when passing back the cached migration version. 2024-06-13 17:50:35 +01:00
mike12345567 3dcce65c9c revert. 2024-06-13 17:50:07 +01:00
mike12345567 8970705b39 Adding a minimum time to the app migration screen and adding a link to documentation. 2024-06-13 17:49:41 +01:00
Sam Rose 1161c185e2
Down to 0 failures. 2024-06-13 17:46:03 +01:00
Dean 3a10c57651 Merge commit 2024-06-13 16:56:25 +01:00
Sam Rose c01c2c7cc3
Down to 4 failures. 2024-06-13 16:23:16 +01:00
Sam Rose 746ee711ae
Down to 19 failures. 2024-06-13 14:45:43 +01:00
Sam Rose f352c5efc1
Merge branch 'master' of github.com:budibase/budibase into budi-8349-in-memory-search-parity-testing 2024-06-13 12:24:40 +01:00
Sam Rose cb6acd8c0b
Down to 71 failures. 2024-06-13 12:24:27 +01:00
Sam Rose 7e69f85e77
More progress. 2024-06-12 18:07:46 +01:00
Sam Rose 22bf0d05ad
Making progress. 2024-06-12 17:58:13 +01:00
Sam Rose ae6539161f
Get rid of negation in predicate. 2024-06-12 17:28:03 +01:00
Michael Drury c4e305e0a0
Merge branch 'master' into fix/app-migrations-automation-service 2024-06-12 15:33:52 +01:00
Sam Rose 7e4f571eb3
wip 2024-06-12 15:24:46 +01:00
Sam Rose 95f5844a44
Get in-memory searching into the search tests. 2024-06-12 15:24:46 +01:00
mike12345567 bf75d9da80 Fix lint. 2024-06-12 15:21:00 +01:00
Sam Rose c759d7b63a
Remove more instances of the work lucene. 2024-06-12 15:13:50 +01:00
Sam Rose 781f3bd64a
Merge branch 'master' of github.com:budibase/budibase into budi-8338-refactor-in-memory-filter-naming 2024-06-12 15:06:22 +01:00
mike12345567 585bbefb90 Moving app migrations back to the automation service now that its getting updated in QA - also making sure that the app migration is always cached, disabling it in dev/QA may hide issues with caching, making sure it always runs the same as it does in prod is safest. 2024-06-12 15:04:18 +01:00
melohagan f8765fb254
Don't add or remove quotas for external DB rows (#13926)
* Don't add or remove quotas for external DB rows

* update account-portal

* fix unit test
2024-06-12 13:49:37 +00:00
Sam Rose bc8791e91e
Remove the word 'lucene' from runLuceneQuery and buildLuceneQuery. 2024-06-12 14:44:20 +01:00
Sam Rose a140a002e2
Merge branch 'master' of github.com:budibase/budibase into fix/aws-session-token-s3-ver2 2024-06-11 18:04:40 +01:00
Michael Drury 3e209abe87
Merge branch 'master' into lock-container-start 2024-06-11 17:53:28 +01:00
Sam Rose 96efb17678
Fix lint. 2024-06-11 17:52:02 +01:00
Sam Rose 85c59c0350
Changing tactic to relying on stable container names to prevent duplication. 2024-06-11 17:41:48 +01:00
mike12345567 c2415012c2 Keep QA check for no caching of app migration info. 2024-06-11 16:02:38 +01:00
Sam Rose 6a54b58303
? 2024-06-11 16:00:04 +01:00
mike12345567 2b2079b4f1 Moving app migrations back to API service. 2024-06-11 15:45:07 +01:00
Sam Rose af60ff4da7
Can't use retries with the sync API. 2024-06-11 15:43:12 +01:00
Sam Rose eac6106b06
Try the sync versions? 2024-06-11 15:38:25 +01:00
Sam Rose 2e67ae115e
Attempt to use unluck we get back from lock. 2024-06-11 15:33:18 +01:00
Sam Rose af0802df30
Only hold the lock during container start. 2024-06-11 14:43:45 +01:00
Sam Rose 1d1ca694c8
Lock starting containers. 2024-06-11 14:39:44 +01:00
mike12345567 334334bbcd Adding a check for QA. 2024-06-11 12:13:18 +01:00
mike12345567 66789888ac Caching is making testing difficult - want to confirm app migrations are doing what they are expected to do 2024-06-11 11:23:52 +01:00
mike12345567 d0736cbe9e Missing next(). 2024-06-10 23:11:53 +01:00
mike12345567 553c2186b1 Only try to lookup migrations if there are migrations to work with. 2024-06-10 23:01:39 +01:00
mike12345567 8c1735a1bd Adding concurrency, and changing how context is set. 2024-06-10 22:58:28 +01:00
mike12345567 a40baf5111 Merge branch 'master' of github.com:Budibase/budibase into fix/some-potential-app-migration-issues 2024-06-10 22:43:49 +01:00
mike12345567 4c873b9921 Attempting to fix some potential app migration issues around versions. 2024-06-10 22:38:16 +01:00
Michael Drury cede6a8a0b
Merge branch 'master' into fix/aws-session-token-s3-ver2 2024-06-10 21:56:50 +01:00
mike12345567 b286e2340b Fixing an issue with test build. 2024-06-10 21:48:02 +01:00
mike12345567 739ac5d03c Putting a better startup process in place for app migrations and adding them to bullboard as well. 2024-06-10 21:37:01 +01:00
mike12345567 1a0d6ef5b0 Only run app migrations in API service - testing this in QA. 2024-06-10 20:46:08 +01:00
mike12345567 834de1f64a Log every step of the app migration process for easier visibility. 2024-06-10 19:19:43 +01:00
mike12345567 c328dd5cd4 Making sure any error that occurs in the app migration system gets logged. 2024-06-10 18:32:06 +01:00
mike12345567 3542c37af0 Merge branch 'fix/aws-session-token-s3-ver2' of github.com:Budibase/budibase into fix/aws-session-token-s3-ver2 2024-06-10 17:18:48 +01:00
mike12345567 8665737dcb Bringing app migration tests back. 2024-06-10 17:18:27 +01:00
Sam Rose a89b1b27e7
Merge branch 'master' into fix/aws-session-token-s3-ver2 2024-06-10 17:05:52 +01:00
Sam Rose 13ec629109
Trigger server tests. 2024-06-10 15:50:04 +01:00