Commit Graph

670 Commits

Author SHA1 Message Date
mike12345567 343ff1271d Required work to support the new lucene audit logs search indexing. 2023-02-17 16:26:55 +00:00
mike12345567 ee70944176 Updating events to include a proper friendly map of audited events. 2023-02-15 19:44:32 +00:00
mike12345567 fd41acf984 Adding in event auditing to server/worker. 2023-02-15 18:03:55 +00:00
Rory Powell 3da6502c8c Merge master into develop 2023-02-15 15:10:02 +00:00
mike12345567 91df9ffbc1 Removing pro from backend-core. 2023-02-15 13:49:30 +00:00
mike12345567 a190bd3da7 Update on audit logs, getting the full write flow from events configured. 2023-02-15 12:37:32 +00:00
mike12345567 7bc850f6d8 Some updates to add in the audit log DB. 2023-02-13 18:16:13 +00:00
mike12345567 7cd28ca4d3 Moving lucene handling to backend-core so that it can be used for other databases (outside row indexes). 2023-02-13 17:13:59 +00:00
Rory Powell 91e63c8ec4 Rename LOG_4XX to ENABLE_4XX_HTTP_LOGGING and enable by default 2023-02-13 14:39:24 +00:00
Rory Powell c78b21ef5f
Merge pull request #9669 from Budibase/budi-6558-configurable-test-log-levels-and-common
Configurable test log levels and common error handling
2023-02-13 14:31:43 +00:00
Rory Powell d68880665c
Merge pull request #9670 from Budibase/budi-6559-enable-higher-concurrency-and-resiliency
Enable higher concurrency and resiliency in worker tests
2023-02-13 14:31:14 +00:00
Rory Powell f9c60d2d11 Add LOG_4XX to environment 2023-02-13 12:27:49 +00:00
Rory Powell 3f0f5b7346 Enable use of redis container in worker tests 2023-02-13 12:09:16 +00:00
Rory Powell 1dc3d44a44 Enable higher concurrency and resiliency in worker tests 2023-02-13 11:57:30 +00:00
Rory Powell d99794c040 Configurable test log levels and common error handling 2023-02-13 11:53:01 +00:00
mike12345567 8a6a10a398 Merge branch 'develop' of github.com:Budibase/budibase into fix/cli-memory-issues 2023-02-07 09:56:49 +00:00
mike12345567 7722bf8d87 Some fixes for #8770 - the CLI was very memory intensive when working with large databases, that contained many revisions. We don't need the revisions for app exports/backups, therefore I've updated our export systems to drop the revision history as it compacts the database significantly and speeds up exports/reduces memory usage. 2023-02-06 17:54:40 +00:00
Adria Navarro 95ada723ce Don't log on CI 2023-02-04 21:04:40 +00:00
adrinr 06ffca74cd Cleanup 2023-02-03 19:23:36 +00:00
adrinr c747cabe73 Rename name for id 2023-02-03 12:09:56 +00:00
adrinr 6d3117d6ac Dry 2023-02-03 11:28:27 +00:00
adrinr 9034a7f7e8 Merge branch 'develop' into feature/dependencies-image 2023-02-03 10:44:40 +00:00
adrinr f70a3d095c Clean code 2023-02-03 09:37:34 +00:00
Dean f51fa2e18a Merge remote-tracking branch 'origin/develop' into fix/add-onboarding-tour-tenant-flag 2023-02-02 11:38:57 +00:00
Dean 04386d01c2 Added new unit tests for feature flags 2023-02-02 11:38:10 +00:00
adrinr 13db0d20e2 Undone user cache changes 2023-02-01 17:20:48 +00:00
adrinr 8589ee24a8 Remove remaning faker refs 2023-02-01 17:19:56 +00:00
adrinr aa8aa2b311 Replace faker for chance 2023-02-01 17:11:50 +00:00
Adria Navarro 25fb740aef
Merge branch 'develop' into feature/dependencies-image 2023-02-01 16:32:00 +00:00
Dean 7242dd4162 Updated default feature flags to include onboarding by default. Added tenant overrides to allow exclusion from global features 2023-02-01 14:12:43 +00:00
mike12345567 a71ee1b693 PR comments. 2023-02-01 13:01:38 +00:00
adrinr 13a1b15f93 Increase timeout 2023-02-01 12:29:03 +00:00
adrinr 4865feeeca More seq 2023-02-01 11:46:31 +00:00
adrinr 4b1d47b900 Merge branch 'develop' into feature/dependencies-image 2023-02-01 10:56:22 +00:00
mike12345567 c038e483e2 Updates to use our new Nano layer for CouchDB integration rather than PouchDB. 2023-01-31 19:49:31 +00:00
adrinr 057ec7d540 Fix installation race conditions 2023-01-31 17:27:25 +00:00
deanhannigan 7a1464bb9d
Merge pull request #9409 from Budibase/feature/user-onboarding-overlays
Feature/user onboarding overlays
2023-01-31 16:37:18 +00:00
adrinr 6090ed36f2 Handle db creation race conditions 2023-01-31 16:15:11 +00:00
Adria Navarro 35a92a5fad
Merge branch 'develop' into feature/dependencies-image 2023-01-30 18:54:05 +00:00
adrinr 3df4c553e7 Run conflicting tests in sequential mode 2023-01-30 13:35:09 +00:00
adrinr d6da46a079 Use test-containers for backend-core tests 2023-01-27 11:43:36 +00:00
adrinr e11f1d1d44 Fix issues with parallel tests and self/cloud modes 2023-01-26 18:20:21 +00:00
Adria Navarro Redo bf0f7d5bab Fix user test 2023-01-26 18:20:21 +00:00
Adria Navarro Redo 498d1776e0 Fix snapshots test 2023-01-26 18:20:21 +00:00
Adria Navarro Redo d46ef8e68c Fix application delete tests 2023-01-26 18:20:21 +00:00
Adria Navarro 730c40e6b9 tweak tests 2023-01-24 15:46:50 +00:00
Adria Navarro aafc70dcff Remove test logic from getAllDbs 2023-01-24 14:06:28 +00:00
mike12345567 1387eedf7a Adding events for environment variables (created, deleted, upgrade panel opened). 2023-01-23 18:18:06 +00:00
Adria Navarro b0ffdc079d Merge branch 'feature/test-image' into feature/dependencies-image 2023-01-23 18:08:19 +00:00
Adria Navarro 963791bc12 Add jest-testcontainers-config 2023-01-23 15:52:23 +00:00
Dean 103bdeaf53 Merge remote-tracking branch 'origin/develop' into feature/user-onboarding-overlays 2023-01-23 09:34:20 +00:00
Dean 37a454a7d3 Added new onboarding tour flow for builder 2023-01-23 00:32:01 +00:00
mike12345567 8a7922110d Merge branch 'develop' of github.com:Budibase/budibase into feature/test-image 2023-01-20 19:55:11 +00:00
mike12345567 20a677e510 Merge branch 'develop' of github.com:Budibase/budibase into feature/environment-variables 2023-01-17 15:25:17 +00:00
Gerard Burns 36f9bd8d71
JSON import changes (#9288)
* JSON import changes

* Fix lockfiles

* PR Feedback, fix tests

* fix tests

* pr feedback
2023-01-17 15:07:52 +00:00
mike12345567 258c5ea957 Some type updates and an improvement to encryption to allow selecting the secret from an option list. 2023-01-16 18:15:43 +00:00
mike12345567 186aeeec2d Adding the ability to store environment variables to context, to make them more easily available/re-usable, as well as exposing them to queries. 2023-01-13 19:53:46 +00:00
Dean 9aeca2ad08 Merge remote-tracking branch 'origin/cheeks-lab-day-portal-redesign' into feature/user-onboarding-overlays 2023-01-13 09:22:41 +00:00
Dean 5ceac908cc Merge commit 2023-01-13 09:20:55 +00:00
Peter Clement 410a4ac3b2 Merge remote-tracking branch 'origin/develop' into feature/environment-variables 2023-01-12 14:54:02 +00:00
Rory Powell 3d115829d1
Fix pino request id generation (#9324)
Remove the duplicated pino config in favour of the common config. Move request id generation into common configuration
2023-01-12 10:25:21 +00:00
Rory Powell 34ee521961
Logging type updates to fix account portal build (#9317) 2023-01-12 08:37:41 +00:00
Rory Powell 6b2b46cc4a
Correlation id's and log context (#7232)
* Correlation id's and log context

* fix build

* Remove redundant fields from LoggingProcessor.ts

* Only log correlation id when present e.g. this is not present on startup / shutdown / automations
2023-01-11 20:39:33 +00:00
mike12345567 73884a9a6f Merge branch 'develop' of github.com:Budibase/budibase into feature/environment-variables 2023-01-11 18:08:49 +00:00
mike12345567 a7bcfe387c Adding in basic implementation of variable usage, getting from pro and enriching through new datasource SDK. 2023-01-11 17:57:51 +00:00
mike12345567 9482870d21 Merge branch 'develop' of github.com:Budibase/budibase into feature/test-image 2023-01-11 10:31:57 +00:00
Rory Powell 13bfd5d8ff
Prefer app url resolution over referer (#9218)
* Prefer app url resolution over referer

* Update lockfiles
2023-01-11 10:24:57 +00:00
Rory Powell 0b2a445a9a
Replace ts-jest with swc/jest (#9289)
- Add swc dependencies
- Add transform change to jest.config.ts
- Replace `export =` with `export default` in src code to enable code coverage to work with swc transformer
- Restructure backend-core errors package to allow for exporting error classes with ESM syntax
- Update backend-core to no longer use `export =`, export individual packages instead of replacing with `export default` for backwards compatibility
- Update event publishers to use `export default` - this was required for the `jest.spyOn` usage inside backend-core common mocks
- Restructure some jest.mock usages where declaring the jest.fn as a variable outside of the package mock threw an error
2023-01-11 09:37:37 +00:00
mike12345567 a0dfe50fcc Merge branch 'develop' of github.com:Budibase/budibase into feature/test-image 2023-01-10 18:49:07 +00:00
mike12345567 8f3e841a75 Licensing updates for environment variables. 2023-01-09 15:42:09 +00:00
Hong Phuc Nguyen af3d736ea3
Doing sanitizeKey when doing deleteFile(s) (#9209)
* Doing sanitizeKey when doing deleteFile(s)

Doing sanitizeKey when doing deleteFile(s). 
In case of backup file name in backups bucket backup-2023-01-03T185512.686Z.tar.gz but in global-db store backup filename is backup-2023-01-03T18:55:12.686Z.tar.gz
sanitizeKey on deleteFile(s) method will prevent all other case related to this bug

* use .promise() instead callback for workaround aws-sdk bug
2023-01-05 08:54:10 +00:00
mike12345567 6518dfbe3b Playing around with getting tests working. 2023-01-04 19:47:56 +00:00
Martin McKeaveney 8a2a5a1f76
fixes for google sheets, admin checklist, and deleting an app from API (#8846)
* fixes for google sheets, admin checklist, and deleting an app from API

* code review

* splitting unpublish endpoint, moving deploy endpoint to applications controller. Still to do public API work and move deployment controller into application controller

* updating REST method for unpublish in API test

* unpublish and publish endpoint on public API, delete endpoint unpublishes and deletes app

* removing skip_setup from prodAppDb call

* removing commented code

* unit tests and open API spec updates

* unpublish, publish unit tests - delete still in progress

* remove line updating app name in API test

* unit tests

* v2.1.46

* Update pro version to 2.1.46

* v2.2.0

* Update pro version to 2.2.0

* Fix for budibase plugin skeleton, which utilises the old import style.

* Fix side nav styles

* v2.2.1

* Update pro version to 2.2.1

* using dist folder to allow importing constants for openAPI specs

* v2.2.2

* Update pro version to 2.2.2

* Fix for user enrichment call (updating to @budibase/nano fork) (#9038)

* Fix for #9029 - this should fix the issue users have been experiencing with user enrichment calls in apps, essentially it utilises a fork of the nano library we use to interact with CouchDB, which has been updated to use a POST request rather than a GET request as it supports a larger set of data being sent as query parameters.

* Incrementing Nano version to attempt to fix yarn registry issues.

* v2.2.3

* Update pro version to 2.2.3

* Fix SQL table `_id` filtering (#9030)

* Re-add support for filtering on _id using external SQL tables and fix filter key prefixes not working with _id field

* Remove like operator from internal tables and only allow basic operators on SQL table _id column

* Update data section filtering to respect new rules

* Update automation section filtering to respect new rules

* Update dynamic filter component to respect new rules

* v2.2.4

* Update pro version to 2.2.4

* lock changes (#9047)

* v2.2.5

* Update pro version to 2.2.5

* Make looping arrow point in right direction (#9053)

* v2.2.6

* Update pro version to 2.2.6

* Types/attaching license to account (#9065)

* adding license type to account

* removing planDuration

* v2.2.7

* Update pro version to 2.2.7

* Environment variable type coercion fix (#9074)

* Environment variable type coercion fix

* Update .gitignore

* v2.2.8

* Update pro version to 2.2.8

* tests passing

* all tests passing, updates to public API response

* update unpublish call to return 204, openAPI spec and unit

* fixing API tests

Co-authored-by: Budibase Release Bot <>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Andrew Kingston <andrew@kingston.dev>
Co-authored-by: melohagan <101575380+melohagan@users.noreply.github.com>
Co-authored-by: Rory Powell <rory.codes@gmail.com>
2022-12-19 13:18:00 +00:00
Rory Powell cfadd037ff
Minio and local licensing fixes (#9071) 2022-12-16 11:14:31 +00:00
Andrew Kingston 20ff5dcb70
Master => Develop (#9049)
* v2.1.46

* Update pro version to 2.1.46

* v2.2.0

* Update pro version to 2.2.0

* Fix for budibase plugin skeleton, which utilises the old import style.

* Fix side nav styles

* v2.2.1

* Update pro version to 2.2.1

* using dist folder to allow importing constants for openAPI specs

* v2.2.2

* Update pro version to 2.2.2

* Fix for user enrichment call (updating to @budibase/nano fork) (#9038)

* Fix for #9029 - this should fix the issue users have been experiencing with user enrichment calls in apps, essentially it utilises a fork of the nano library we use to interact with CouchDB, which has been updated to use a POST request rather than a GET request as it supports a larger set of data being sent as query parameters.

* Incrementing Nano version to attempt to fix yarn registry issues.

* v2.2.3

* Update pro version to 2.2.3

* Fix SQL table `_id` filtering (#9030)

* Re-add support for filtering on _id using external SQL tables and fix filter key prefixes not working with _id field

* Remove like operator from internal tables and only allow basic operators on SQL table _id column

* Update data section filtering to respect new rules

* Update automation section filtering to respect new rules

* Update dynamic filter component to respect new rules

Co-authored-by: Martin McKeaveney <martin@budibase.com>
Co-authored-by: Budibase Release Bot <>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Martin McKeaveney <martinmckeaveney@gmail.com>
2022-12-15 12:13:42 +00:00
Rory Powell 7ef9e9dc51
File authentication using presigned URLs (#8883)
* file auth wip

* Private plugin bucket wip

* Add etag to logo request, fix circular dependency

* Resolve cyclic dependency

* Ensure browser doesn't cache custom component

* Global file fixes

* File fixes / remove permaUrl

* Add ctx local storage

* Enable presigned url to work from any host

* Test fix

* Self hosting w/ S3 + other fixes

* Fix for serving dev app in multi tenant

* Fix backup restore and import attachment url processing, never store attachment urls

* Test fixes

* Unit tests WIP

* Replace file path with base64 encoded private key

* Multi tenancy tests for files

* Environment files fixes

* Post-merge build and test fixes

* Add debug conditional to all jest logging and revert/fix prod minio proxy_pass

* Re-add default '/' minio route :(

* Rename iconFile -> iconFileName

* Add cloudfront vars to k8s templates

* Remove public bucket policy

* Remove duplicated test
2022-12-15 11:35:22 +00:00
Rory Powell 56dfba9b4a
Offline license POC (#9046) 2022-12-14 21:48:48 +00:00
Peter Clement 9bf37b34ae
Server and Worker tests (#8928)
* automation unit tests

* row processor tests

* update executeQuery test

* update groups assertion

* some more worker tests

* plugin tests and tidying up

* linting

* temporarily disable group tests

* more tests

* fix import

* fix backup tests

* add pro mocks to worker

* check for app existence in import test

* test new tsconfig

* testing changes

* Pro test utils (#9020)

* Pro test utils changes

* Add test fixes

* Remove ts-ignore

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Rory Powell <rory.codes@gmail.com>
2022-12-12 22:02:32 +00:00
mike12345567 0b7057dd1d Some fixes based on test failure. 2022-12-07 17:33:26 +00:00
Andrew Kingston 02ef58c7fc Merge branch 'develop' of github.com:Budibase/budibase into side-panel 2022-11-29 12:22:56 +00:00
mike12345567 ab914bcca2 Final fixes after re-running all test suites. 2022-11-28 19:12:23 +00:00
mike12345567 b5b74b9cad General work after running some tests in CI - general refactoring as well. 2022-11-28 17:54:04 +00:00
mike12345567 8f4813b01a Fix for account portal. 2022-11-28 14:09:18 +00:00
mike12345567 6ac4669f0a Fixing some issues detected by testing. 2022-11-26 21:26:19 +00:00
mike12345567 4868083208 Fixes based on server build so far. 2022-11-26 15:10:41 +00:00
mike12345567 ea66f7ac38 Disabling js interop in Typescript build of backend-core and worker, fixing build issues. 2022-11-26 14:46:01 +00:00
mike12345567 0a32126c5b Automation overhaul to Typescript, plus type updates. 2022-11-25 19:57:07 +00:00
Andrew Kingston 2a6ba0536f Merge branch 'develop' of github.com:Budibase/budibase into side-panel 2022-11-25 11:24:57 +00:00
mike12345567 a38d617138 Complete conversion of backend-core to Typescript. 2022-11-24 18:48:51 +00:00
Andrew Kingston 92987ebfb8 Rename migration 2022-11-24 14:27:49 +00:00
mike12345567 1b408e60f2 Refactoring worker, converting all controllers/routes to Typescript and all imports of backend-core to new style. 2022-11-23 18:25:20 +00:00
mike12345567 4fd408defc Merge branch 'develop' of github.com:Budibase/budibase into refactor/backend-core-import 2022-11-23 10:30:57 +00:00
Rory Powell 70016dc7e4 Account portal <-> backend-core fixes 2022-11-22 22:24:45 +00:00
mike12345567 6f6d000cb8 Finally removing all usages of backend-core/ type imports from server, including some further typescript conversions. 2022-11-22 19:49:59 +00:00
mike12345567 a620791d13 More controller typescript conversions. 2022-11-22 16:52:25 +00:00
Andrew Kingston cac780eb50 Merge branch 'develop' of github.com:Budibase/budibase into side-panel 2022-11-22 16:52:09 +00:00
Rory Powell b7d33dfa5f Fix view creation 2022-11-22 16:28:51 +00:00
mike12345567 7ab2029b5d Typescript conversion of the table controllers. 2022-11-22 13:56:01 +00:00
mike12345567 51471449fc Merge branch 'develop' of github.com:Budibase/budibase into refactor/backend-core-import 2022-11-22 12:03:31 +00:00
mike12345567 ee4ebbfb56 Some typescript conversions, moving a few imports around. 2022-11-21 18:33:34 +00:00