Commit Graph

72 Commits

Author SHA1 Message Date
mike12345567 0efa1f06ab Moving some stuff around to make way for other services using the sql layers. 2024-05-16 17:33:47 +01:00
mike12345567 aa51db20ee Changing how cleanup works - the cleanup is now part of the DB deletion, making sure it cannot be missed. 2024-05-14 15:41:48 +01:00
Peter Clement a4c0328c53
REST file handling and SMTP automation block attachments (#13403)
* handle files in rest connector

* fetch presigned url and return

* further updates to handle files in rest connector

* remove unused important and fix extension bug

* wrong expiry param

* tests

* add const for temp bucket

* handle ttl on bucket

* more bucket ttl work

* split out fileresponse and xmlresponse into utils

* lint

* remove log

* fix tests

* some pr comments

* update function naming and lint

* adding back needed response for frontend

* use fsp

* handle different content-disposition and potential path traversal

* add test container for s3 / minio

* add test case for filename* and ascii filenames

* move tests into separate describe

* remove log

* up timeout

* switch to minio image instead of localstack

* use minio image instead of s3 for testing

* stream file upload instead

* use streamUpload and update signatures

* update bucketcreate return

* throw real error

* tidy up

* pro

* pro ref fix?

* pro fix

* pro fix?

* move minio test provider to backend-core

* update email builder to allow attachments

* testing for sending files via smtp

* use backend-core minio test container in server

* handle different types of url

* fix minio test provider

* test with container host

* lint

* try different hostname?

* Revert "try different hostname?"

This reverts commit cfefdb8ded.

* fix issue with fetching of signed url with test minio

* update autoamtion attachments to take filename and url

* fix tests

* pro ref

* fix parsing of url object

* pr comments and linting

* pro ref

* fix pro again

* fix pro

* account-portal

* fix null issue

* fix ref

* ref

* When sending a file attachment in email fetch it directly from our object store

* add more checks to ensure we're working with a signed url

* update test to account for direct object store read

* formatting

* fix time issues within test

* update bucket and path extraction to regex

* use const in regex

* pro

* Updating TTL handling in upload functions (#13539)

* Updating TTL handling in upload functions

* describe ttl type

* account for ttl creation in existing buckets and update types

* fix tests

* pro

* pro
2024-04-22 15:30:57 +00:00
mike12345567 6e4a66b2e1 Initial implementation of generating SQS junction table definitions. 2024-04-11 18:19:47 +01:00
Adria Navarro 4e53cb5143 Flags 2024-03-01 13:09:42 +01:00
mike12345567 21a8d44ec7 Adding a 'SESSION_EXPIRY_SECONDS' environment variable which can be set on the services to configure how long before an idle user is logged out. 2024-01-25 16:22:39 +00:00
mike12345567 f2f16cfcee Adding an option to disable password validation when creating an admin user - this means that the environment variables used for BB_ADMIN creation can have any length of password (not breaking change). 2024-01-05 13:17:10 +00:00
Sam Rose 6b491815a5
Add isSqsAvailable key to the environment endpoint. 2023-12-11 15:22:02 +00:00
Adria Navarro 3c85f1ce9e Fix after renames 2023-10-23 13:55:58 +02:00
Adria Navarro 4ee1fd0f7a Default to package.json 2023-10-23 12:23:44 +02:00
Adria Navarro 961913b88a Use version from environment 2023-10-20 16:21:35 +02:00
Adria Navarro 205d33f172 Clean cypress refs 2023-09-07 15:37:22 +02:00
mike12345567 3a211b8865 Some builder test cases, check the service type switch works as expected. 2023-07-28 16:50:44 +01:00
mike12345567 f061426b60 Fixing build issue. 2023-07-28 16:08:33 +01:00
mike12345567 72438f531d Update for worker, get builder endpoints working for app builders. 2023-07-28 15:39:59 +01:00
Rory Powell 5ac024a655 Merge branch 'develop' into feature/offline-license 2023-07-20 12:27:04 +01:00
Adria Navarro 252ee9d27c Dropping the default values 2023-07-11 14:02:18 +02:00
Adria Navarro dc73e57ba8 Respect max values 2023-07-11 13:48:52 +02:00
Adria Navarro e351a77f75 Decorate service 2023-07-11 10:44:09 +02:00
Adria Navarro b0f2306c50 Fix types 2023-07-11 10:44:09 +02:00
Adria Navarro ef0888fcad Read package name 2023-07-11 10:44:09 +02:00
Rory Powell 586bca16d0 Move OFFLINE_MODE to backend-core environment 2023-07-06 20:44:48 +01:00
Michael Drury a3f9759a47 Reverting ioredis-mock removal for test cases - will come back to this in future as it currently breaks pro tests. 2023-06-01 00:02:55 +01:00
mike12345567 249625ad1a Updating backend tests to actually utilise the redis container which is spun up as part of the tests, doing away with the somewhat problematic ioredis-mock which was breaking some tests due to the addition of the changes for multi-dev collab. 2023-05-31 17:18:01 +01:00
mike12345567 1aad88adc8 Fix for CLI startup complaining about version not being found. 2023-05-15 15:52:59 +01:00
Martin McKeaveney b387f969d9 allow passwordless redis 2023-05-08 15:59:12 +01:00
Martin McKeaveney 50de32a8a0 Merge pull request #10452 from Budibase/feature/clustering-support
attaching clustered mode to an env var
2023-05-04 14:25:23 +01:00
mike12345567 552499b781 Re-writing the disabling of pino/logging - it seems that the pino logger is causing a variety of issues in the built CLI version - easier to offer an environment variable for backend-core which completely removes the logger. 2023-05-02 16:22:43 +01:00
Martin McKeaveney e75e91e248 control all redis clustering through env var 2023-04-28 19:53:15 +01:00
Martin McKeaveney f73af919a7 attaching clustered mode to an env var 2023-04-28 19:09:22 +01:00
adrinr 8e95121ac8 Implement find version 2023-04-17 14:07:49 +01:00
adrinr e5227febd8 Use version from environment 2023-04-17 14:02:50 +01:00
Rory Powell 976b3a55ca Update logging to support dd trace attributes (#10086)
* Update logging middleware to integrate with pino for console logging

* Remove elastic apm references, use updated core middlewares

* Remove redundant LOG_LEVEL definitions

* Remove no longer needed jest logging overrides

* lint

* Backwards compat between console log helpers and pino

* Configurable DISABLE_HTTP_LOGGING

* Don't log 4xx as errors

* Remove redundant ENABLE_4XX_HTTP_LOGGING

* Cleanup migrations and event logging

* Improve bb-alert logging

* Add DISABLE_HTTP_LOGGING to helm chart

* Add ops endpoints for testing

* Disable http logging in dev

* Backwards compatible tracing implementation

* Naming update on http logging env var

* lint

* Update packages/backend-core/src/environment.ts

Co-authored-by: Adria Navarro <adria@revityapp.com>

* Merge

* Lint

* Fix console.warn failing mock by replacing with alerts mock instead

* Lint

---------

Co-authored-by: Adria Navarro <adria@revityapp.com>
2023-04-04 15:08:46 +01:00
Michael Drury 34e8009f08 Merge pull request #9949 from Budibase/fix/blacklist
Blacklisting
2023-03-20 10:33:11 +00:00
Rory Powell e116941750 Rotatable secrets (#9982)
* Rotatable secrets

* Set new api encryption key var

* Lint

* Use fallback keys instead of array

* Point api encryption key to dedicated value

* Add API_ENCRYPTION_KEY to cli

* Lint + add api encryption key to env files
2023-03-13 15:02:59 +00:00
mike12345567 564c724be1 Provides the option for blacklisting IPs. 2023-03-08 16:30:26 +00:00
mike12345567 04ef7eeb9c Minor fixes - after removing axios altogether and adding a start script to make life easier for running from ts. 2023-03-03 10:03:33 +00:00
mike12345567 2fb33e2cf3 Merge branch 'develop' of github.com:Budibase/budibase into feature/audit-logs 2023-02-27 14:51:33 +00:00
Rory Powell 2c46109e7d Enforceable SSO (#9787)
* Add ENFORCEABLE_SSO feature flag

* First draft of enforce sso configuration / show single sign on url

* Reading and writing isSSOEnforced + integration with login page

* Enable CI + lint

* Set correct base branch for CI

* Test fix for expected string changed

* Use tenant aware platform url as SSO link

* Bring in latest pro changes

* Lint

* Add useEnforceableSSO mock helper function

* Update configs.spec.ts with coverage for public settings

* Update users.spec.ts with additional tests for isPreventPasswordActions

* Lint

* Update refresh OAuthToken to use correct enum and add case statement
2023-02-27 13:42:51 +00:00
mike12345567 0fa984f26f Moving is audited and adding in env var for ip address/user agent auditing. 2023-02-27 11:42:52 +00:00
Rory Powell d3a7286711 Configs updates: remove circular deps, dedicated module, typing improvements, reduce db reads 2023-02-23 13:41:35 +00:00
Rory Powell cc7eb64a3b Rename LOG_4XX to ENABLE_4XX_HTTP_LOGGING and enable by default 2023-02-13 14:39:24 +00:00
Rory Powell 0e3a17ab18 Merge pull request #9669 from Budibase/budi-6558-configurable-test-log-levels-and-common
Configurable test log levels and common error handling
2023-02-13 14:31:43 +00:00
Rory Powell f53faff7ad Add LOG_4XX to environment 2023-02-13 12:27:49 +00:00
Rory Powell 07e5598538 Enable use of redis container in worker tests 2023-02-13 12:09:16 +00:00
mike12345567 c645a9bc21 Some type updates and an improvement to encryption to allow selecting the secret from an option list. 2023-01-16 18:15:43 +00:00
Rory Powell fba7ef26ad Replace ts-jest with swc/jest (#9289)
- Add swc dependencies
- Add transform change to jest.config.ts
- Replace `export =` with `export default` in src code to enable code coverage to work with swc transformer
- Restructure backend-core errors package to allow for exporting error classes with ESM syntax
- Update backend-core to no longer use `export =`, export individual packages instead of replacing with `export default` for backwards compatibility
- Update event publishers to use `export default` - this was required for the `jest.spyOn` usage inside backend-core common mocks
- Restructure some jest.mock usages where declaring the jest.fn as a variable outside of the package mock threw an error
2023-01-11 09:37:37 +00:00
Rory Powell 786039055e File authentication using presigned URLs (#8883)
* file auth wip

* Private plugin bucket wip

* Add etag to logo request, fix circular dependency

* Resolve cyclic dependency

* Ensure browser doesn't cache custom component

* Global file fixes

* File fixes / remove permaUrl

* Add ctx local storage

* Enable presigned url to work from any host

* Test fix

* Self hosting w/ S3 + other fixes

* Fix for serving dev app in multi tenant

* Fix backup restore and import attachment url processing, never store attachment urls

* Test fixes

* Unit tests WIP

* Replace file path with base64 encoded private key

* Multi tenancy tests for files

* Environment files fixes

* Post-merge build and test fixes

* Add debug conditional to all jest logging and revert/fix prod minio proxy_pass

* Re-add default '/' minio route :(

* Rename iconFile -> iconFileName

* Add cloudfront vars to k8s templates

* Remove public bucket policy

* Remove duplicated test
2022-12-15 11:35:22 +00:00
Rory Powell f935f05509 Offline license POC (#9046) 2022-12-14 21:48:48 +00:00
Peter Clement 9d841bc947 Server and Worker tests (#8928)
* automation unit tests

* row processor tests

* update executeQuery test

* update groups assertion

* some more worker tests

* plugin tests and tidying up

* linting

* temporarily disable group tests

* more tests

* fix import

* fix backup tests

* add pro mocks to worker

* check for app existence in import test

* test new tsconfig

* testing changes

* Pro test utils (#9020)

* Pro test utils changes

* Add test fixes

* Remove ts-ignore

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Rory Powell <rory.codes@gmail.com>
2022-12-12 22:02:32 +00:00