Commit Graph

1129 Commits

Author SHA1 Message Date
melohagan be0ff766f3
Chore/fix worker request error message (#12943)
* Revert "Surface email error (#12837)"

This reverts commit 2a24c85378.

* Improve workerRequest error handling

* update account portal

* Fix typing

* lint
2024-02-02 15:25:18 +00:00
Adria Navarro 8dfcc3aeb1
Revert "Revert "Migrate from `vm` to `isolated-vm`"" 2024-02-02 12:02:09 +01:00
Sam Rose 332a201ae7
Revert "Migrate from `vm` to `isolated-vm`" 2024-02-02 09:51:46 +00:00
mike12345567 6cad0e758f Quick fix for automation JS scripting, make sure context is passed through correctly. 2024-02-01 16:00:58 +00:00
Adria Navarro 181d462cdf Update uuid 2024-02-01 11:15:21 +01:00
Adria Navarro 623b4f5dab
Merge branch 'master' into isolated-vm 2024-01-31 10:47:08 +01:00
mike12345567 0eb29ef73f PR comments. 2024-01-24 18:00:32 +00:00
mike12345567 ccc751d09c A quick refactor to get rid of the old 'FieldTypes' enumeration, considering how core it is to all data handling in Budibase, the fact we had both 'FieldType' and 'FieldTypes' was confusing - righting this fully. 2024-01-24 16:58:13 +00:00
Adria Navarro a65892d29b
Merge branch 'master' into isolated-vm 2024-01-23 17:32:54 +01:00
melohagan 2a24c85378
Surface email error (#12837) 2024-01-22 15:30:21 +00:00
Peter Clement 50078f4f0f Fix issue where flagClient could be undefined when running outside of test mode 2024-01-19 14:59:08 +00:00
Sam Rose 58abca62de
work in progress: replace vm2 and vm with isolated-vm 2024-01-05 13:48:20 +00:00
Sam Rose 642b75e0ae
Revert "Remove all custom tracing to see if it's the cause of the memory leak." 2024-01-02 11:36:32 +00:00
Sam Rose 3c0901f530
Remove all custom tracing to see if it's the cause of the memory leak. 2023-12-21 11:06:05 +00:00
Sam Rose c4f124c78a
Update utils.ts
Co-authored-by: Adria Navarro <adria@budibase.com>
2023-12-15 12:22:37 +00:00
Sam Rose 7a11c5be92
Tweak newly added spans to track formula processing. 2023-12-15 11:10:23 +00:00
Sam Rose a6f03b02f2
Use the right casing for DataDog 2023-12-14 17:27:22 +00:00
Sam Rose 899b6707e7
add the column into the processStringSync span 2023-12-14 17:10:09 +00:00
Sam Rose 0d3ea23301
Instrument formula processing in DataDog. 2023-12-14 16:52:47 +00:00
mike12345567 d4a4bb96f4 PR comments 2023-12-08 13:24:24 +00:00
mike12345567 f6d5db7d61 Adding test cases for attachment cleanup update. 2023-12-08 13:21:08 +00:00
mike12345567 7665d2fd53 Fix for attachment deletion when updating table column name, as well as refactoring to make the code a bit easier to follow. 2023-12-07 16:35:43 +00:00
mike12345567 936ce9fcc2 Adding test to make sure attachments are output correctly. 2023-12-06 12:29:37 +00:00
mike12345567 be6cb0825c Fixing an issue with import and images not displaying correctly. 2023-12-05 18:27:56 +00:00
melohagan 0e93717f1a
Allow null strings (#12298)
* Allow null strings

* Don't update null to blank

* Save empty as null

* Make blank string map to null

* Add mappings for BigInt

* Fix unit tests
2023-11-23 09:53:35 +00:00
Michael Drury 1d368be5de
Merge branch 'master' into fix/admin-user-backup 2023-11-22 10:59:45 +00:00
mike12345567 18a275ecb1 Removing unused function. 2023-11-21 17:44:46 +00:00
mike12345567 ddd8482013 Updating bb admin user creation so that it can be used incase in self host a user gets locked out, the environment variables can be used to create a simple user to access the system. 2023-11-21 17:30:11 +00:00
Adria Navarro db2c20e3ae eslint import/newline-after-import 2023-11-20 21:52:29 +01:00
Adria Navarro 92f411bf77 Prettier 2023-11-20 21:48:49 +01:00
Adria Navarro 4d27ff1dc0 eslint import/no-duplicates 2023-11-20 21:48:49 +01:00
mike12345567 dde446286d Switching getMultiple to default to failure if not all entries found, then updating usages. 2023-11-09 17:08:14 +00:00
mike12345567 a26f2e83e4 Some of the new table getters did not account for table IDs not being found, adding a new function which properly accounts for IDs being missing and manages it correctly, rather than leaving tables in an undefined state. 2023-11-09 14:45:23 +00:00
mike12345567 4dc6b1eeda Merge branch 'master' of github.com:Budibase/budibase into fix/11972-external-relationship-formulas 2023-11-08 14:41:53 +00:00
Sam Rose 0633a3de65
WIP 2023-11-08 10:03:22 +00:00
Sam Rose fdfda100c1
Ensure that the DB always returns Documents. 2023-11-07 18:14:52 +00:00
mike12345567 312415ca7d Enforcing squash for rows which contain circular structures. 2023-11-07 13:47:21 +00:00
mike12345567 49b00ee7ee Fixing build issue. 2023-11-07 11:07:25 +00:00
mike12345567 635af0f76a Getting external rows to be more consistent with the internal API - the formulas should be processed using the outputProcessing as they were being processed, but was causing an overwrite. The problem was that the external system internally processed the formulas/relationships, then squashed the relationships. Once it got to the external API, it would go through normal output processing, which would run over the squashed rows, causing an inconsistent behaviour. 2023-11-06 19:03:37 +00:00
Michael Drury 9643d9c0e2
Merge branch 'master' into feature/budi-7607-migrate-user-relationship-columns-to-the-new-user-column 2023-10-30 17:41:44 +00:00
Samuel-Martineau 674f40a06e
Fix how attachment URLs are handled 2023-10-30 11:12:16 -04:00
mike12345567 9b1a7bd854 PR comments - updating pickAPI to use a common function. 2023-10-27 11:43:01 +01:00
mike12345567 19eaafd946 Fixing some test cases. 2023-10-26 16:32:34 +01:00
mike12345567 ed0670a008 Major update to make the table.type always 'table' and then adding a new sourceType which states what source the table came from, external or internal. Don't want to keep using a type that should be static as two different things. 2023-10-26 13:19:09 +01:00
mike12345567 455b26bac9 Making sure the source ID is always set when creating a table - the frontend expects this to be set for every table so making the type represent this correctly. 2023-10-25 19:00:25 +01:00
mike12345567 13dadbcc7d Merge branch 'master' of github.com:Budibase/budibase into fix/internal-db-enrich-perf 2023-10-18 15:22:00 +01:00
Sam Rose 767900aec6
Remove type information from JSDoc in .ts files, it's redundant. 2023-10-17 16:46:32 +01:00
Michael Drury a23f918880 Merge branch 'master' into fix/internal-db-enrich-perf 2023-10-12 11:09:17 +01:00
Adria Navarro 5908a791f2 Return email and names on bb reference api output 2023-10-11 19:10:58 +02:00
mike12345567 09ce950e64 Merge branch 'master' of github.com:Budibase/budibase into fix/internal-db-enrich-perf 2023-10-11 17:24:08 +01:00
Adria Navarro 62fd9011f3 Merge branch 'labday/type_schema' into feature/multi-user-type-column 2023-10-11 14:13:55 +02:00
Adria Navarro 715db97436 Clean code 2023-10-11 13:21:51 +02:00
Adria Navarro 9391854f46 Types 2023-10-11 13:21:19 +02:00
Adria Navarro 6fa5727166 Clean 2023-10-11 13:19:05 +02:00
Adria Navarro f57104730e Remove subtype from the base field schema 2023-10-11 12:23:54 +02:00
Adria Navarro a8c814e714 Fixes fixes 2023-10-10 18:54:57 +02:00
Adria Navarro 6c328109b6 Fixes 2023-10-10 18:54:57 +02:00
Adria Navarro 21e2d7ddbe Fix relationship types 2023-10-10 18:54:57 +02:00
Adria Navarro c8ffa98844 Fixes 2023-10-10 18:54:57 +02:00
Adria Navarro 530857a6ff Fix 2023-10-10 15:39:05 +02:00
Adria Navarro c49a16fdac Fix tests 2023-10-10 15:35:00 +02:00
Adria Navarro 21420c59fa Fix order 2023-10-10 14:39:55 +02:00
Adria Navarro 78335be26a Clean 2023-10-10 14:20:26 +02:00
Adria Navarro d3e8e1f4f8 Fix importing 2023-10-10 13:52:47 +02:00
Adria Navarro b966ed221d Fix bulk delete 2023-10-10 13:52:47 +02:00
Adria Navarro fd414dbe4b Enable importing bbrefs as string csv 2023-10-10 13:52:47 +02:00
Adria Navarro 979c2aeb00 Return only id and primary display 2023-10-10 13:52:47 +02:00
Adria Navarro df97b7ad93 Expect BB ref data as csv strings 2023-10-10 13:52:47 +02:00
Adria Navarro f22467fa53 Validate user column 2023-10-10 13:52:47 +02:00
Adria Navarro 7d6cd53c29 Add comments 2023-10-09 15:37:42 +02:00
Adria Navarro 0909e0547e Process formulas after processing complex fields 2023-10-09 15:30:07 +02:00
Adria Navarro 047936bcbf Merge branch 'master' into budi-7589/user-column-multi-user-filtering-support 2023-10-09 09:22:55 +02:00
mike12345567 0dc28122fa Adding test case. 2023-10-06 16:38:36 +01:00
mike12345567 d4335bca73 Quick fix for saving current user._id - make sure it is the correct format of ID. 2023-10-06 16:31:48 +01:00
mike12345567 3e2f9dfc4e Further enhancement, client library sends up the column it wants enriched and then we can ignore everything else, makes a big difference for enriching users (with a lot of relationships). 2023-10-05 18:23:18 +01:00
Adria Navarro 46b85ac12c Multiple users column 2023-10-04 17:55:23 +02:00
Adria Navarro 753cb442c2 Allow edit 2023-10-04 17:55:23 +02:00
Adria Navarro 1a00b60088 Fix tests 2023-09-29 16:39:18 +02:00
Adria Navarro 7050c5352a Check external table types 2023-09-29 16:09:08 +02:00
Adria Navarro 9d186df018 Don't modify internal API 2023-09-29 14:58:02 +02:00
mike12345567 5a3ecc3095 Getting create/update response consistent with internal API for external. 2023-09-29 13:40:40 +01:00
Adria Navarro fad4f12e26 Fix test 2023-09-28 10:00:25 +02:00
Adria Navarro fdb6474868 Fix tests 2023-09-28 10:00:25 +02:00
Adria Navarro 9afa334501 Fix deletions 2023-09-28 10:00:25 +02:00
Michael Drury a67f78fead Merge branch 'develop' into fix/BUDI-7236 2023-09-27 17:08:21 +01:00
Martin McKeaveney 65af2ed7c2 backmerge from master 2023-09-27 17:06:57 +01:00
mike12345567 bb2892cbc2 Getting client library loading in Webstorm debug, as well as adding accessible roles endpoint. 2023-09-27 14:21:25 +01:00
Adria Navarro 38d6fa8b2a Renames 2023-09-22 17:33:13 +02:00
Adria Navarro 8a5270c6ee Fix 2023-09-20 13:19:38 +02:00
mike12345567 65d96b5ec5 Adding message to let user know why roles have been ignored, as well as adding test case for this. 2023-09-20 12:13:10 +01:00
Adria Navarro 31b29662d6 Handle delete behaviours 2023-09-20 12:47:47 +02:00
Adria Navarro 244af30b6a Fix tests 2023-09-20 12:47:47 +02:00
Adria Navarro af499412cf Don't return the full user object 2023-09-20 12:47:47 +02:00
Adria Navarro d1cfb64017 Renames 2023-09-20 10:08:47 +02:00
Adria Navarro ed830835a4 use bulk cache in processOutputBBReferences 2023-09-20 10:07:38 +02:00
Adria Navarro 58d9d3e8e8 use bulk cache in processInputBBReferences 2023-09-20 10:05:52 +02:00
Adria Navarro 1314b991c7 Value is always csv 2023-09-19 13:17:07 +02:00
Adria Navarro 89af863c34 Lint 2023-09-19 10:14:25 +02:00
Adria Navarro 256ab3a296 Fix test 2023-09-18 12:22:18 +02:00
Adria Navarro ead2a2795e Fix deletion 2023-09-18 11:58:34 +02:00
Adria Navarro 7f6ef551c9 Return always arrays for bb processor fields 2023-09-18 11:57:27 +02:00
Adria Navarro 60d94e76cf Lint 2023-09-15 13:49:24 +02:00
Adria Navarro 29eaeacc4c Fix deletions 2023-09-15 13:47:08 +02:00
Adria Navarro 385989eca4 Process output 2023-09-15 13:31:56 +02:00
Adria Navarro bebe342b33 Handle output processing 2023-09-15 13:31:22 +02:00
Adria Navarro 9510c37b7a Handle bb ref only if there is a value 2023-09-15 13:04:45 +02:00
Adria Navarro 679f56f32f Error handling 2023-09-15 12:29:57 +02:00
Adria Navarro d62e9a475f Validate objects 2023-09-15 12:10:59 +02:00
Adria Navarro 0d3f9dac8c Validate and return csv 2023-09-15 12:07:25 +02:00
Adria Navarro 1ff3f5db80 Handle ids 2023-09-15 11:21:10 +02:00
Adria Navarro 4c4c2e118b Renames 2023-09-15 10:54:43 +02:00
Adria Navarro 8a9474c2ac Clean test 2023-09-15 10:44:50 +02:00
Adria Navarro e36fd75bc1 Add tests 2023-09-15 10:41:39 +02:00
Adria Navarro 0e3f316eb2 Process bb ref on input processing 2023-09-15 10:33:36 +02:00
Adria Navarro edd8879d04 inputProcessing async 2023-09-15 10:31:54 +02:00
Adria Navarro 1cbfeafe39 Remove context refs in row processor 2023-09-15 09:56:28 +02:00
Adria Navarro a012c0a458 Add back base default legacy views permissions 2023-09-04 12:00:58 +02:00
mike12345567 3395a5b96b Some other minor changes to fully support the per app builder from groups, making sure middlewares are properly aware. 2023-08-22 19:15:47 +01:00
mike12345567 656870db8b Adding last of support for per app group builder support, enriching the user on self return, as well as adding the functionality required to server middlewares. 2023-08-22 18:14:08 +01:00
mike12345567 a5a1c62eb1 Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7108 2023-08-18 14:34:08 +01:00
mike12345567 7110351bdc Revert "revert develop into master"
This reverts commit fd099658a0.
2023-08-18 14:33:21 +01:00
mike12345567 621b1fadea Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7108 2023-08-18 13:22:34 +01:00
Martin McKeaveney fd099658a0 revert develop into master 2023-08-17 10:28:15 +01:00
mike12345567 5d2813361c Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7108 2023-08-14 11:00:06 +01:00
mike12345567 2011e1693e PR comments. 2023-08-08 12:06:25 +01:00
mike12345567 4c2e3a5489 Updating last remaining view perms to table perms. 2023-08-07 16:49:13 +01:00
mike12345567 a2667c6d72 Removing the ability to set roles, builder and admin structure through basic public API. 2023-08-04 15:43:02 +01:00
Adria Navarro e011ccd0c4 Merge branch 'develop' into chore/typecheck_tests 2023-08-01 16:49:06 +01:00
Adria Navarro c6c1450d06 Fixe types 2023-08-01 12:52:12 +02:00
mike12345567 8522cb94d3 Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7052 2023-07-28 17:10:43 +01:00
Adria Navarro b9820f94e6 Import lodash modules 2023-07-28 17:11:48 +02:00
mike12345567 3798caf86d Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7052 2023-07-28 16:08:38 +01:00
Adria Navarro 2c6725404f Types 2023-07-26 13:54:35 +02:00
mike12345567 87d80e08f5 Merge branch 'develop' of github.com:Budibase/budibase into feature/BUDI-7052 2023-07-21 17:34:27 +01:00
mike12345567 bb032852e5 Merge branch 'master' of github.com:Budibase/budibase into develop 2023-07-21 16:26:34 +01:00
Andrew Kingston 08920f8015 Remove invalid columns from display column selection and fix schema failure handling 2023-07-20 14:06:31 +01:00
Andrew Kingston 5d0918a6cb Prevent invalid characters in column names when importing tables 2023-07-20 12:21:09 +01:00
mike12345567 39746e0bf0 Main body of work to handle the new approach of per app builders support. 2023-07-18 16:57:48 +01:00
Adria Navarro 2698e47ec4 Type internal db.get 2023-07-18 11:41:51 +02:00
Michael Drury e1741937ae Removing logging of error in coercion - this was creating noise in errors channel. 2023-06-21 12:08:39 +01:00
mike12345567 cbfeea121c Merge branch 'master' of github.com:Budibase/budibase into develop 2023-06-20 18:09:23 +01:00
melohagan 9e9cece59e Clone user (#10974) 2023-06-20 17:07:34 +01:00
Adria Navarro dbcf7814a8 Support ssl 2023-06-19 17:56:11 +01:00
Rory Powell 5f716a62a4 Revert "Merge branch 'develop' into master"
This reverts commit 67fb9de7f4, reversing
changes made to ebec885b0a.
2023-06-16 10:13:43 +01:00
melohagan c0578d4cc2 Send meeting invite via automation email smtp (#10787)
* Pass calendar props into sendSmtpEmail

* Add calendar event to message

* Add Checkbox and DatePicker automation field UI

* Add URL prop

* Add url to sendSmtpEmail unit test

* Refactor

* Code review comments

* Make location optional

* Add EmailInvite type

---------

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
2023-06-08 14:25:35 +01:00
mike12345567 8d7119a363 Make sure top level path is resolved correctly. 2023-06-07 16:37:42 +01:00
Rory Powell abbd8c5c8b Merge branch 'master' into develop 2023-06-05 23:19:43 +01:00
Andrew Kingston 6f51843875 Fix TS issue 2023-06-02 16:22:07 +01:00
Michael Drury 8e4f0e531c Merge pull request #10755 from Budibase/fix/datasource-info-endpoint
Datasource info endpoint update
2023-06-01 12:29:46 +01:00
deanhannigan a782a8b9b1 Merge pull request #10758 from Budibase/fix/reduce-array-row-parse-log-level
Reduced log level of array parsing on row import
2023-06-01 10:41:19 +01:00
Dean 6ee83e2da8 Reduced log level of parse array attempt 2023-06-01 09:05:22 +01:00
Michael Drury a3f9759a47 Reverting ioredis-mock removal for test cases - will come back to this in future as it currently breaks pro tests. 2023-06-01 00:02:55 +01:00
mike12345567 039e78379f Fixing lint issue. 2023-05-31 17:48:18 +01:00
mike12345567 fba3ec987e Merge branch 'develop' of github.com:Budibase/budibase into fix/datasource-info-endpoint 2023-05-31 17:37:17 +01:00
mike12345567 249625ad1a Updating backend tests to actually utilise the redis container which is spun up as part of the tests, doing away with the somewhat problematic ioredis-mock which was breaking some tests due to the addition of the changes for multi-dev collab. 2023-05-31 17:18:01 +01:00
Andrew Kingston 8a2d4b7052 Merge branch 'develop' of github.com:Budibase/budibase into websocket-redis-integration 2023-05-31 16:28:14 +01:00
Andrew Kingston 05e34076f7 Fully rewrite websocket redis integration to use individual keys per session, enable TTLs on sesisons, prune sessions when users connect and add a heartbeat to sockets 2023-05-31 15:13:22 +01:00
Andrew Kingston d926650232 Add back in functionality to select different redis DB's per client 2023-05-31 10:52:39 +01:00
Andrew Kingston d3fdb52bf5 Store all socket users in redis to allow all server instances to be aware of all users 2023-05-31 10:21:50 +01:00
mike12345567 7c7bd4d5cb Fix for debugging with webstorm the old way (if desired), updating the builder middleware to be more multi-dev capable, ignoring 409s when attempting to update the last updated at for apps (if multiple devs hit at same time, only use one) also updating writethrough cache to retry once, with the extended TTL on locks, plus the multi-dev collab it can take a minute to update usage quota doc when a lot of updates occur at once. 2023-05-30 17:41:20 +01:00
Andrew Kingston 5e480a1527 Use socker.io redis adapter to broadcast events to all server instances 2023-05-25 08:48:56 +01:00
Adria Navarro 3d7af9b2a3 Fix manifest ref 2023-05-22 09:42:43 +02:00
Adria Navarro 986d4c4740 Merge branch 'develop' into chore/esbuild 2023-05-19 17:26:47 +02:00
Adria Navarro 82f53483b2 Fix server docker 2023-05-19 16:44:09 +02:00
Adria Navarro 0a2335a047 Merge pull request #10622 from Budibase/chore/improve_pipeline_times
Chore - Improve pipeline run time
2023-05-19 13:18:21 +02:00
mike12345567 cf7107d19a Updating sync to cover both prod and dev apps. 2023-05-17 19:35:00 +01:00
mike12345567 f3785892a9 Merge branch 'develop' of github.com:Budibase/budibase into chore/esbuild 2023-05-17 12:45:28 +01:00
Adria Navarro 60109ebcda Fix manifest usage 2023-05-17 12:55:04 +02:00
Dean 1e6652dc39 Fix for importing exported array/option fields. Fix to ensure lastid and inclusion updates persisted as a result of an import. Test updates for array and option fields 2023-05-10 12:36:01 +01:00
Dean b0ff61c9ec Feedback update. Console.error to logging.logalert 2023-05-05 09:24:57 +01:00
Dean 5498cc86bd Refactor to fix tests and added an extra empty scenario 2023-05-04 12:58:45 +01:00
Dean a1fdff8df8 Linting 2023-05-04 12:25:30 +01:00
Dean 56a1b068cc Added json parse option for row import attachment values 2023-05-04 11:52:20 +01:00
Adria Navarro efaf2adf05 Fix build 2023-05-03 18:05:32 +01:00
Adria Navarro 1d041a3dd5 Support nulls when optional 2023-05-02 12:48:05 +01:00
Adria Navarro 6e7c78362e Return null instead of undefined 2023-05-02 12:46:53 +01:00
Adria Navarro 650cbc1f01 Handle cast as undefineds 2023-05-02 11:57:18 +01:00
Adria Navarro c87cc39cea Add basic tests 2023-05-02 10:44:25 +01:00
Adria Navarro 29df12c247 Add csvutils 2023-05-02 10:34:45 +01:00
mike12345567 4f7e56fed9 Making build script into a module, so that it can be used in a specialised server build which also builds the thread scripts as required for worker-farm. 2023-04-27 19:00:01 +01:00
Michael Drury 9c93bf91c0 Merge pull request #10352 from Budibase/fix/10349
Allow SQL formulas using related row information to be used as display columns
2023-04-24 17:50:49 +01:00
mike12345567 1aca5d6407 Fixing an issue with creating relationships between existing tables, not using the correct type for the primary key in the foreign key relationship. 2023-04-19 17:17:37 +01:00
Michael Drury 42c37eea46 Merge pull request #10276 from Budibase/labday/automation-plugins
Automation step plugins
2023-04-18 17:33:25 +01:00
mike12345567 23e99ca4b7 Adding group and user tests for user sync. 2023-04-12 19:59:05 +01:00
mike12345567 ac37d00f69 Updating plugins to have a SDK, which can be used for automation action retrieval as well as datasources. 2023-04-11 16:37:26 +01:00
mike12345567 e699f4684a Updating the global user sync to be more accurate and also remove old user metadata from apps that users don't have access to anymore. 2023-04-04 18:03:56 +01:00
mike12345567 2e80bab7ad Removing requirement for built client library in component test case. 2023-03-31 17:13:25 +01:00
mike12345567 906d614c64 Merge branch 'merge-master-develop' of github.com:Budibase/budibase into merge-master-develop 2023-03-31 13:51:20 +01:00
Rory Powell 44aa19a99a Merge branch 'master' into merge-master-develop 2023-03-31 12:26:04 +01:00
Rory Powell 79bec7d86d Add force exit, -e flag and streaming to tests (#10151)
* Add force exit, -e flag and streaming to tests

* Temp remove script usage from worker tests

* Revert "Temp remove script usage from worker tests"

This reverts commit 5e0ed2e92d.

* Always fail test

* Ignore server test package from coverage report

* Unscope server only

* Run all server tests again

* Update backend core test script to fail when any command fails and force exit

* Don't try to require client in test

* Port memory limit fix from develop

* Remove always failing test and commented out block for client require

* Fixing some issues with test cases.

* Revert assertions in internalSearch.spec.js

---------

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
2023-03-31 10:32:13 +01:00
adrinr 8d45e44e2f Fix types 2023-03-30 11:30:35 +01:00
mike12345567 1c73b92595 Fix for app sync, base it on group roles, not just user roles - stops app sync from pulling in group users which do not actually have access to the app. 2023-03-21 13:55:28 +00:00
Rory Powell e116941750 Rotatable secrets (#9982)
* Rotatable secrets

* Set new api encryption key var

* Lint

* Use fallback keys instead of array

* Point api encryption key to dedicated value

* Add API_ENCRYPTION_KEY to cli

* Lint + add api encryption key to env files
2023-03-13 15:02:59 +00:00
Peter Clement 1cb03c7603 fix roleId being undefined leading to groups override 2023-03-07 13:30:24 +00:00
Rory Powell 940de8b6a0 Run CI steps in parallel (#9760)
* Parallel CI

* Add build to integration test

* Add checkout to top of each run

* Revert branch update for ci job

* Experiment with --runInBand for CI

* Fix intermittent backend-core migration test failure

* Fix hanging worker redis connection

* Update naming from reset to newTenant
2023-02-21 17:13:24 +00:00
Adria Navarro 16e53957bc Merge branch 'develop' into feature/dependencies-image 2023-01-30 18:54:05 +00:00
adrinr d160430ac8 Start race condition 2023-01-30 18:51:22 +00:00
mike12345567 3659768c42 Merge branch 'develop' of github.com:Budibase/budibase into feature/environment-variables 2023-01-17 15:25:17 +00:00
Gerard Burns 48e1695318 JSON import changes (#9288)
* JSON import changes

* Fix lockfiles

* PR Feedback, fix tests

* fix tests

* pr feedback
2023-01-17 15:07:52 +00:00
Peter Clement 117aaed356 Merge remote-tracking branch 'origin/develop' into feature/environment-variables 2023-01-12 14:54:02 +00:00
Rory Powell e5e767f18d Correlation id's and log context (#7232)
* Correlation id's and log context

* fix build

* Remove redundant fields from LoggingProcessor.ts

* Only log correlation id when present e.g. this is not present on startup / shutdown / automations
2023-01-11 20:39:33 +00:00
mike12345567 d27674de5c Replacing some require statements with imports. 2023-01-11 19:07:01 +00:00
Rory Powell fba7ef26ad Replace ts-jest with swc/jest (#9289)
- Add swc dependencies
- Add transform change to jest.config.ts
- Replace `export =` with `export default` in src code to enable code coverage to work with swc transformer
- Restructure backend-core errors package to allow for exporting error classes with ESM syntax
- Update backend-core to no longer use `export =`, export individual packages instead of replacing with `export default` for backwards compatibility
- Update event publishers to use `export default` - this was required for the `jest.spyOn` usage inside backend-core common mocks
- Restructure some jest.mock usages where declaring the jest.fn as a variable outside of the package mock threw an error
2023-01-11 09:37:37 +00:00
Rory Powell 786039055e File authentication using presigned URLs (#8883)
* file auth wip

* Private plugin bucket wip

* Add etag to logo request, fix circular dependency

* Resolve cyclic dependency

* Ensure browser doesn't cache custom component

* Global file fixes

* File fixes / remove permaUrl

* Add ctx local storage

* Enable presigned url to work from any host

* Test fix

* Self hosting w/ S3 + other fixes

* Fix for serving dev app in multi tenant

* Fix backup restore and import attachment url processing, never store attachment urls

* Test fixes

* Unit tests WIP

* Replace file path with base64 encoded private key

* Multi tenancy tests for files

* Environment files fixes

* Post-merge build and test fixes

* Add debug conditional to all jest logging and revert/fix prod minio proxy_pass

* Re-add default '/' minio route :(

* Rename iconFile -> iconFileName

* Add cloudfront vars to k8s templates

* Remove public bucket policy

* Remove duplicated test
2022-12-15 11:35:22 +00:00
Peter Clement 9d841bc947 Server and Worker tests (#8928)
* automation unit tests

* row processor tests

* update executeQuery test

* update groups assertion

* some more worker tests

* plugin tests and tidying up

* linting

* temporarily disable group tests

* more tests

* fix import

* fix backup tests

* add pro mocks to worker

* check for app existence in import test

* test new tsconfig

* testing changes

* Pro test utils (#9020)

* Pro test utils changes

* Add test fixes

* Remove ts-ignore

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Rory Powell <rory.codes@gmail.com>
2022-12-12 22:02:32 +00:00
mike12345567 b9ce140d95 Fix for #7732 - as well as some tests for it, make sure that it is working as expected. 2022-12-06 17:20:26 +00:00
Michael Drury a656c6e678 Adding test case for char encoding and being explicit about utf8 export. 2022-11-29 16:03:22 +00:00
Michael Drury 31fbc937b0 Removing old JS file. 2022-11-29 15:23:01 +00:00
Michael Drury bcf2aadbaa Merge branch 'develop' of github.com:Budibase/budibase into fix/charset-encoding 2022-11-29 11:57:45 +00:00
mike12345567 12d6e60525 One file left in the server, some fixes after converting the internal row controller. 2022-11-26 15:42:53 +00:00
mike12345567 658f12281d Fixes based on server build so far. 2022-11-26 15:10:41 +00:00
mike12345567 a3bb2e0d77 Automation overhaul to Typescript, plus type updates. 2022-11-25 19:57:07 +00:00
mike12345567 7eac8fafd6 Some server typescript re-writes, only automations left and some controllers. 2022-11-25 15:01:46 +00:00
mike12345567 514f9a1210 Some fixes based on test runs. 2022-11-23 11:44:15 +00:00
mike12345567 02ede5d067 Finally removing all usages of backend-core/ type imports from server, including some further typescript conversions. 2022-11-22 19:49:59 +00:00
mike12345567 8c2d9ebec8 Updating all of the route files to typescript, as well as some controllers. 2022-11-22 18:49:19 +00:00
mike12345567 8a4da7d4ce Typescript conversion of the table controllers. 2022-11-22 13:56:01 +00:00
mike12345567 4cdcafac36 More Typescript conversion, as part of backend-core importing improvements. 2022-11-22 12:41:36 +00:00
mike12345567 de82eca502 Some typescript conversions, moving a few imports around. 2022-11-21 18:33:34 +00:00
mike12345567 31c198888a Updating some enums, plural to single. 2022-11-17 14:59:18 +00:00
mike12345567 4328ed1b67 Updating middlewares to Typescript, as well as some fixes based on running tests. 2022-11-16 17:24:13 +00:00
mike12345567 535fab7997 Some major reworks towards higher levels of typescript. 2022-11-16 17:23:12 +00:00
mike12345567 5f658a0398 Converting further DB layers to TS - fixing mocks. 2022-11-15 18:33:24 +00:00
mike12345567 b5472f882b Some updates for currentapp.spec.js test case. 2022-11-15 17:35:17 +00:00
Rory Powell 998beddc30 Configurable CDN 2022-11-07 09:45:00 +00:00
mike12345567 11115aab56 Merge branch 'develop' of github.com:Budibase/budibase into feature/app-backups 2022-10-17 14:49:31 +01:00
Dean 7125e3f6e3 Merge remote-tracking branch 'origin/develop' into feature/qr-barcode-reader 2022-10-13 16:16:28 +01:00
mike12345567 9d34ccfaca Merge branch 'develop' of github.com:Budibase/budibase into feature/app-backups 2022-10-12 11:59:00 +01:00
Rory Powell 2f0f36e635 Merge branch 'master' into merge-master 2022-10-12 10:19:46 +01:00
mike12345567 19133f08e6 Adding import functionality - still need to update the attachments URL. 2022-10-11 20:25:22 +01:00
mike12345567 7c71f76b70 Export to tarball through tmp. 2022-10-11 18:21:58 +01:00
Rory Powell ef4b1dd32b Fix user metadata update 2022-10-11 12:42:35 +01:00
mike12345567 1f36eec89a Some updates towards supporting attachments in app exports. 2022-10-10 20:08:59 +01:00
Dean 07be0208e5 Merge remote-tracking branch 'origin/develop' into feature/qr-barcode-reader 2022-10-10 09:08:48 +01:00
mike12345567 2d4eb10aba Merge branch 'develop' of github.com:Budibase/budibase into feature/app-backups 2022-10-07 20:48:28 +01:00
mike12345567 197d195cb4 Switching from scan for app locks to mget - which is a fast O(N) operation that only retrieves what we need. 2022-10-07 18:05:18 +01:00
Michael Drury 060b2aba3c Merge pull request #8175 from Budibase/fix/remove-app-scans
Removing Redis SCAN operation from app services
2022-10-07 17:35:05 +01:00
mike12345567 0c4a40c795 Switching from scan for app locks to mget - which is a fast O(N) operation that only retrieves what we need. 2022-10-07 16:05:01 +01:00
Dean c09154b03e Renamed the 'Scanned Code' field type to 'Barcode/QR'. The component is also now listed as 'Barcode/QR Scanner'. Minor fix to include longform text columns in the table csv import list 2022-10-07 11:00:25 +01:00
mike12345567 ef52bde670 Building out initial SDK work - converting some existing exporting work to typescript. 2022-10-06 19:10:45 +01:00
mike12345567 ea2fb6de1b Merge branch 'master' of github.com:Budibase/budibase into develop 2022-10-06 16:54:25 +01:00
Dean e251c914ff Review feedback changes. Changed 'Code' labelling to 'Scanned Code' 2022-10-06 13:02:17 +01:00
Dean 2ec21741d1 Initial commit of QR Reader field 2022-10-05 09:28:07 +01:00
Andrew Kingston f2f842444a Update plugin URL utility export to use CJS syntax 2022-10-05 08:57:26 +01:00
Andrew Kingston 788dac0f8d Use CJS style imports in plugin URL helper to work in jest, and respect plugin bucket env var 2022-10-05 08:25:50 +01:00
Andrew Kingston b487855348 Update plugin JS URL handling 2022-10-04 16:28:21 +01:00
Andrew Kingston de9b2c44cb Add utility for generating plugin JS URLs 2022-10-04 16:27:04 +01:00
Rory Powell 79226777b1 Merge develop 2022-09-28 08:56:45 +01:00
Martin McKeaveney c309d25b16 Merge pull request #7881 from Budibase/feature/email-cc
Add CC and BCC (with multiple email support) to Send Email automation action step
2022-09-25 13:40:33 +01:00
Michael Drury c7fb092294 Fixing issue discovered by test. 2022-09-23 22:24:31 +01:00
mike12345567 e0b6ef66ac Fixing a few issues with roles being correctly reverted. 2022-09-23 17:45:26 +01:00
mike12345567 9b29bd6709 Merge branch 'develop' of github.com:Budibase/budibase into group-fixes 2022-09-22 18:44:16 +01:00
mike12345567 e3f054470a Updating a few core endpoints to better integrate the groups system and make sure users always have the correct role ID updated onto them. 2022-09-22 18:27:43 +01:00
Mel O'Hagan 1b49f02092 CC and BCC interface 2022-09-21 15:58:04 +01:00
Martin McKeaveney 9b5ed59f34 Merge pull request #7792 from Budibase/feature/plugin-pro
Plugin pro integration
2022-09-18 10:34:06 +01:00
mike12345567 6bad41e3ce Minor fix - datasource hot reloading was working apart from the the CJS require caching, need to clear down the cache entry before reloading. 2022-09-16 11:25:28 +01:00
Martin McKeaveney c86bdffc52 Merge pull request #7646 from Budibase/qa/qa-core-jest
QA Core repo, and Jest API tests running end to end
2022-09-15 21:12:45 +01:00
mike12345567 40688f6783 Adding budibase/pro integration for plugins, controlling quota limits. 2022-09-15 14:57:27 +01:00
Rory Powell 0f98f2fcaa Merge branch 'master' into develop 2022-09-15 10:19:56 +01:00
Martin McKeaveney ddc585fad3 Merge branch 'develop' into qa/qa-core-jest 2022-09-15 01:18:58 +01:00
mike12345567 416a2c999f Fixing issue with plugin import for datasources - extract the integration correctly. 2022-09-13 12:16:00 +01:00
mike12345567 722a087af5 Reverting organisation page changes. 2022-09-12 16:21:47 +01:00
mike12345567 828066de46 Merge branch 'develop' of github.com:Budibase/budibase into feature/plugin-management-ui 2022-09-12 15:34:34 +01:00
mike12345567 bbed4cce74 Fixing an intermettient test issue. 2022-09-12 15:16:31 +01:00
Rory Powell 45a43baee2 Merge branch 'master' into develop 2022-09-12 14:05:36 +01:00
mike12345567 3d66e71d7d Fixing an issue with deleting large apps with a lot of internal table data and adding back the export modal which allows picking whether an export includes internal table rows - #7583. 2022-09-09 20:06:29 +01:00
mike12345567 8e56a7a4d9 Fixing an issue with external tables containing time only fields. 2022-09-07 17:57:02 +01:00
NEOLPAR c2bca8a025 uploading plugin for github, npm and url 2022-09-06 16:28:35 +01:00
Martin McKeaveney c3f15b5af2 QA Core repo, and Jest API tests running end to end 2022-09-05 18:28:53 +01:00
NEOLPAR 4c47c3a418 sorting npm stuff out 2022-09-05 15:04:26 +01:00
NEOLPAR 7d8940f0cb Merge branch 'plugins-dev-experience' into feature/plugin-management-ui 2022-09-05 09:38:24 +01:00
mike12345567 a5f475872d Updating plugins to not think about versions, only ever one version of each plugin by name, making plugins self host only and adding error checking for datasource implementations. 2022-09-02 19:32:15 +01:00
NEOLPAR 54eed13541 create plugin github public and private 2022-09-01 20:04:45 +01:00
NEOLPAR 0e2a86460c uploading npm and url plugins 2022-08-31 16:09:47 +01:00
mike12345567 320cbfada3 Merge branch 'develop' of github.com:Budibase/budibase into plugins-dev-experience 2022-08-31 15:53:32 +01:00
Rory Powell 5f022e04a3 Merge branch 'develop' into user-fixes 2022-08-31 11:39:41 +01:00
Rory Powell 435465f93a Update shutdown logging 2022-08-31 11:05:49 +01:00
NEOLPAR e5446d2f8a plugin create npm 2022-08-30 21:37:08 +01:00
Martin McKeaveney 02c93071ee merge 2022-08-30 10:33:37 +01:00
Martin McKeaveney 4aa99336d7 adding path variable 2022-08-25 11:07:35 +01:00
Martin McKeaveney d5c2f32df3 additional logging around component definitions 2022-08-25 10:57:30 +01:00
Andrew Kingston 975afc29d2 Merge branch 'plugins-dev-experience' of github.com:Budibase/budibase into plugins-dev-experience 2022-08-22 11:27:18 +01:00
Andrew Kingston 23a91b89a7 Merge branch 'develop' of github.com:Budibase/budibase into plugins-dev-experience 2022-08-22 11:27:04 +01:00
Rory Powell 917bb96ae6 add on() to mock queue 2022-08-20 11:42:51 +01:00
mike12345567 f329f9868c Adding management of the new README.md file that is retrieved with the plugin skeletons. 2022-08-18 18:23:07 +01:00
mike12345567 2bb32253d8 Adding in hash handling for datasource plugins. 2022-08-18 15:21:55 +01:00
mike12345567 7dfe049ae6 Adding full builder integration of custom components - splitting into own section properly - fixing some issues with SVG class overlap. 2022-08-17 23:13:51 +01:00
mike12345567 b6a5ccbf10 Adding some controls around datasource config creation and updating minio fetch. 2022-08-17 11:01:25 +01:00
mike12345567 2aaff7576b Adding full flow for creating custom datasource. 2022-08-17 10:05:13 +01:00
Mel O'Hagan 5171fc09fb Delete attachments on field clear 2022-08-12 11:29:57 +01:00
mike12345567 76d427001a Merge branch 'develop' of github.com:Budibase/budibase into plugins-dev-experience 2022-08-11 16:34:01 +01:00
mike12345567 fc6111272e Fixing viewname enumeration. 2022-08-11 14:40:17 +01:00
mike12345567 b803a3fd93 Replacing all plural TS enumerations with singular - this involves a major rename of the DocumentTypes. 2022-08-11 13:50:05 +01:00
mike12345567 a683665a99 Adding plugin upload API - takes a file form-data and then extracts, uploads to minio and stores data about the plugin to CouchDB. 2022-08-10 20:01:48 +01:00
Martin McKeaveney 7de6117421 changing readstream let to const 2022-08-08 13:54:11 +01:00
Martin McKeaveney 9ec178c7c8 charset encoding 2022-08-08 13:51:02 +01:00
mike12345567 2dea5c5614 Some various session fixes based on current data. 2022-08-05 21:35:26 +01:00
Peter Clement c3a4941119 Merge pull request #6989 from Budibase/pc/fixes
User Management UI fixes
2022-08-01 09:39:55 +01:00
Peter Clement 79f7fd380d pr comments and updating group check to be more safe 2022-07-30 11:38:08 +01:00
mike12345567 07f3897b55 Merge branch 'develop' of github.com:Budibase/budibase into feature/automation-error-stop 2022-07-29 14:31:18 +01:00