Commit Graph

230 Commits

Author SHA1 Message Date
Rory Powell 77ffb8d86d Run integration suite in CI again / auto detect tenancy / refactors (#10209)
* qa-core-ci-fixes

* global setup and teardown wip

* Updates to logs and setup

* Remove date and console mocking

* Update CI to spin up minimal dev env

* Update readme

* Fix scopeBackend.sh

* Ensure docker services are initialised before starting worker

* Lint

* Fix admin user being created on startup (#10219)

* use regular bootstrap and build

* Lint

* Temp: re-use global setup to get around app limit in QA
2023-04-05 15:33:56 +01:00
Rory Powell 976b3a55ca Update logging to support dd trace attributes (#10086)
* Update logging middleware to integrate with pino for console logging

* Remove elastic apm references, use updated core middlewares

* Remove redundant LOG_LEVEL definitions

* Remove no longer needed jest logging overrides

* lint

* Backwards compat between console log helpers and pino

* Configurable DISABLE_HTTP_LOGGING

* Don't log 4xx as errors

* Remove redundant ENABLE_4XX_HTTP_LOGGING

* Cleanup migrations and event logging

* Improve bb-alert logging

* Add DISABLE_HTTP_LOGGING to helm chart

* Add ops endpoints for testing

* Disable http logging in dev

* Backwards compatible tracing implementation

* Naming update on http logging env var

* lint

* Update packages/backend-core/src/environment.ts

Co-authored-by: Adria Navarro <adria@revityapp.com>

* Merge

* Lint

* Fix console.warn failing mock by replacing with alerts mock instead

* Lint

---------

Co-authored-by: Adria Navarro <adria@revityapp.com>
2023-04-04 15:08:46 +01:00
Rory Powell 79bec7d86d Add force exit, -e flag and streaming to tests (#10151)
* Add force exit, -e flag and streaming to tests

* Temp remove script usage from worker tests

* Revert "Temp remove script usage from worker tests"

This reverts commit 5e0ed2e92d.

* Always fail test

* Ignore server test package from coverage report

* Unscope server only

* Run all server tests again

* Update backend core test script to fail when any command fails and force exit

* Don't try to require client in test

* Port memory limit fix from develop

* Remove always failing test and commented out block for client require

* Fixing some issues with test cases.

* Revert assertions in internalSearch.spec.js

---------

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
2023-03-31 10:32:13 +01:00
Rory Powell 9d6d781244 Use `--runInBand` in CI only (#9771)
* Use  in CI only

* Use scripts instead of ternary to pick up correct bash syntax
2023-02-22 13:04:16 +00:00
mike12345567 3f7f66257d Merge branch 'master' of github.com:Budibase/budibase into develop 2023-02-14 17:57:23 +00:00
mike12345567 488b542ac1 Fix for #9654 - the error checking was looking at the through fields flipped, which caused many to many relationships to throw an error. Also I've updated the dev compose file on master so that switching between develop and master is smoother. 2023-02-14 14:38:55 +00:00
mike12345567 3b43943f1f Some quick scripts for loading a lot of rows and apps. 2023-02-03 19:29:28 +00:00
Rory Powell 905de31cd6 Merge pull request #9525 from Budibase/chore/default-local-dev-accounts-disabled
Set DISABLE_ACCOUNT_PORTAL=1 by default in local dev
2023-02-03 09:54:21 +00:00
Rory Powell f28b1dc79a Set DISABLE_ACCOUNT_PORTAL=1 by default in local dev 2023-02-02 16:38:23 +00:00
Dean 0eed617866 Updated default feature flags to include onboarding by default. Added tenant overrides to allow exclusion from global features 2023-02-01 14:12:43 +00:00
mike12345567 95e8371b1d Adding encryption key to manage of server. 2023-01-16 18:51:48 +00:00
Martin McKeaveney 1dcd3445ce adding host.docker.internal mapping to PROXY_ADDRESS 2022-12-21 10:58:39 +00:00
Rory Powell b635614f77 Allow proxy upstream URLs to be configured at runtime (#7413)
* allow proxy upstream urls to be configured at runtime

* make dynamic upstream urls in proxy the default behaviour

* reorder env directives

* Use appVersion for proxy image

* Update couch upstream for proxy

* proxy ci updates - WIP

* allowing custom resolvers in proxy service deployment

* remove generateProxyConf script for prod and remove from builds

* remove generateProxyConfig script

* removing nginx dev conf

* delete generated nginx conf

Co-authored-by: Thomas Weiß <panos@unbunt.org>
Co-authored-by: Martin McKeaveney <martinmckeaveney@gmail.com>
2022-12-20 16:28:13 +00:00
mike12345567 2a51091ffb Adding some null controls. 2022-12-02 16:17:56 +00:00
mike12345567 32488bee88 Adding broken SQL configuration. 2022-12-02 10:50:52 +00:00
mike12345567 02ede5d067 Finally removing all usages of backend-core/ type imports from server, including some further typescript conversions. 2022-11-22 19:49:59 +00:00
Rory Powell 1612044dc1 Merge branch 'develop' into subdomain-tenancy 2022-11-15 16:18:59 +00:00
Rory Powell 9973f2e161 Bump jest and ts-jest to be consistent across backend packages to fix CI issue, test config updates 2022-11-14 22:55:47 +00:00
Rory Powell 5da699e27c Fail pro scripts on error, unify TS versions, update test and coverage configs, use pro source mappings 2022-11-14 13:29:46 +00:00
Rory Powell 6783fd713e Update tenancy detection to honour any subdomain pattern according to platform url 2022-11-09 16:35:16 +00:00
Rory Powell 5d973de503 Add TENANT_FEATURE_FLAGS to default dev environment 2022-09-23 13:48:19 +01:00
Andrew Kingston 5bcb4c5393 Move restart of proxy container to after the compose stack has been brought up to ensure container exists 2022-09-16 09:25:14 +01:00
Andrew Kingston 206714869d Always restart the proxy service when spinning up the dev stack to account for nginx conf changes 2022-09-15 16:31:29 +01:00
Rory Powell a551532e73 Navigation updates, cookie constant, update link-dependencies and localdomain scripts 2022-09-14 10:53:14 +01:00
Andrew Kingston 6093da79d6 Add hot reloading for component plugins via watched plugin directory 2022-08-15 22:23:45 +01:00
Andrew Kingston 467cc27f47 Add env vars to automatically create initial admin user on first run 2022-06-30 11:28:52 +01:00
mike12345567 e32de3b850 Updating install script to work from any directory. 2022-06-21 20:01:50 +01:00
mike12345567 9ad9860a9c Fixing cleanup script, adding ARM support to dockerfile, as well as adding ARM client. 2022-06-21 17:56:55 +01:00
Rory Powell 12aa9fa1ce Dev improvements - remove need to rebuilds of backend-core/dist and enable debugging when running tests and backend services, use source directly using typescript and jest path mappings 2022-06-10 22:57:12 +01:00
Rory Powell 0cf777a0cd Build fixes 2022-06-09 12:33:10 +01:00
Rory Powell a4eef3daab Add awareness of deployment environment 2022-06-01 14:10:00 +01:00
Rory Powell a9ab8b2629 Lots of failure handling and caching 2022-05-30 21:46:08 +01:00
Rory Powell 28b48a88fe Event identification 2022-05-23 22:14:44 +01:00
Rory Powell 70fff5c7b9 Merge branch 'develop' into feature/posthog-v2 2022-05-23 09:04:00 +01:00
mike12345567 c528896e2d Finishing off UI, expanding settings menus from 80ch to 90ch as they were quite restricted and made the usage UI a bit difficult to view. Added a DISABLE_DEVELOPER_LICENSE env variable for testing. 2022-05-17 17:44:40 +01:00
Rory Powell 1caf4b1965 Merge branch 'develop' into feature/posthog-v2 2022-04-27 16:32:00 +01:00
Martin McKeaveney 9ca4dacbaa merge with master 2022-04-26 10:21:45 +01:00
Michael Drury 94419024a4 Merge pull request #5510 from Budibase/fix/pouch-leak
PouchDB memory leak protection
2022-04-22 15:32:51 +01:00
mike12345567 9899781fc5 Some updates, making sure databases will always close directly using finally checks around the actual tasks, updating how replication works to have a close statement (to make sure it is controlled correctly) and then updating to PouchDB 7.3.0 for one of the memory leak fixes. 2022-04-20 17:33:42 +01:00
mike12345567 192fb1307e Initial version of memory leak protection, making sure that PouchDB databases are closed correctly after use, using a combination of closures wrapping DB gets (this replaces the getDB, leaving only a dangerousGetDB function which can be used in very very specific scenarios) and then closing the DB as part of CLS hooked functions finishing. Also moving the GlobalDB init to the tenancy middleware as this is used everywhere in the worker/app services - means that not all getGlobalDB calls require an async closure around them. 2022-04-19 19:42:52 +01:00
Rory Powell 41144db055 Remove remaining direct usage of CouchDB. Add common pouch config to backend-core 2022-04-19 16:45:27 +01:00
mike12345567 5180603374 Fix for #5411 - trimming the various binding components for SQL in queries. Also moving SQL interpolation out into a Typescript - prep for further unit testing/improvements. 2022-04-14 13:06:22 +01:00
Rory Powell 33719fcb09 Merge branch 'feature/licensing' into feature/posthog-v2 2022-04-12 17:12:46 +01:00
Rory Powell 03ba6d914f Merge branch 'develop' into feature/licensing 2022-04-12 16:18:41 +01:00
Rory Powell 5ab9f1a9c5 Table CRUD events + add timekeeper for tests 2022-04-11 14:39:31 +01:00
mike12345567 39182cfbc0 Fixing issue #5817 - making sure that date strings are correctly parsed into the bindings. 2022-04-01 17:13:28 +01:00
Rory Powell 15b676ce1c Remove remaining direct usage of CouchDB. Add common pouch config to backend-core 2022-03-29 16:03:44 +01:00
Rory Powell 75d5c0c0b9 Merge branch 'feature/licensing' into feature/posthog-v2 2022-03-28 09:47:34 +01:00
Rory Powell 69b2b38aea Merge branch 'master' into merge-master 2022-03-25 13:10:37 +00:00
Martin McKeaveney 14c75b2907 update always to unless-stopped 2022-03-24 17:35:35 +00:00
mike12345567 d64df5fd78 Adding a fix for #4996 - checks if WSL is in use when configuring nginx. 2022-03-23 15:31:08 +00:00
Rory Powell 715d42d3e6 Merge branch 'develop' into feature/licensing 2022-03-21 17:13:16 +00:00
Martin McKeaveney 45f158dcc5 Merge pull request #4587 from CommanderRoot/rm-deprecated-substr
Replace deprecated String.prototype.substr()
2022-03-10 01:57:57 +01:00
mike12345567 7f9334ae5e Making sure that tables are created within the correct schema for MS-SQL. 2022-03-04 15:50:19 +00:00
mike12345567 a4ed8fe5e8 Adding the ability to support schemas in MS-SQL. 2022-03-04 15:12:07 +00:00
Rory Powell a81041bc40 Add shared licensing cache 2022-03-03 13:37:04 +00:00
Martin McKeaveney b28bffbb15 Merge pull request #4631 from Budibase/ci/deploy-proxy
removing need for handlebars in nginx script, automating proxy servic…
2022-03-03 09:06:49 +00:00
mike12345567 1b45a9190d Fixes for SQL issues raised by Marty. 2022-03-02 17:40:50 +00:00
mike12345567 8b6a9b7273 Merge branch 'develop' of github.com:Budibase/budibase into feature/budibase-api 2022-02-28 18:43:45 +00:00
mike12345567 c1a1c02e73 Updating validators, make sure everything that is optional is really optional. 2022-02-25 11:39:46 +00:00
Rory Powell 83d8c70419 Change ports 2022-02-23 07:00:57 +00:00
Martin McKeaveney 1cbca0578a removing need for handlebars in nginx script, automating proxy service deployment 2022-02-22 13:24:21 +01:00
Rory Powell b9306b5a7a Prevent local port conflicts between account portal and cypress 2022-02-22 11:20:58 +00:00
Tobias Speicher 491bcb86ed Replace deprecated String.prototype.substr()
String.prototype.substr() is deprecated (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/substr) so we replace it with slice() which works similarily but isn't deprecated.
Signed-off-by: Tobias Speicher <rootcommander@gmail.com>
2022-02-20 15:28:39 +01:00
Rory Powell 02c0c49594 Deprovision and local couch db url fix 2022-02-18 17:36:23 +00:00
Martin McKeaveney a7a82dc679 adding k8s resolver and updating watchtower variable 2022-02-16 15:33:57 +01:00
Martin McKeaveney 97bf3e68dd adding watchtower service 2022-02-16 14:29:08 +01:00
Martin McKeaveney a85f667dbe templatize nginx configuration into one prod configuration and add NPM scripts to manage 2022-02-16 14:12:53 +01:00
Martin McKeaveney a495050871 merge with develop 2022-02-10 11:39:15 +01:00
mike12345567 d13985af3f Fix for #3758 - updating the datetime cell renderer and form component to be capable of display time only formats, as well as adding an option to only select the time component for a date time form field. 2022-02-08 16:15:08 +00:00
mike12345567 f61e15594b Fix for #3928 - error invalid use of FETCH was based on a lack of sorting on MS-SQL platforms, this enforces a sort of some type no matter what. 2022-02-04 16:17:36 +00:00
mike12345567 168a126e2a Updating the SQL core to allow it to handle multiple relationships between the same two tables. 2022-02-02 18:15:17 +00:00
Martin McKeaveney c11e26cb2e merge 2022-02-01 11:22:14 +01:00
Martin McKeaveney 7fb9b33f6c remove more references to envoy 2022-02-01 11:02:37 +01:00
Martin McKeaveney b521721642 fixing cypress setup 2022-01-31 16:35:23 +01:00
Martin McKeaveney cca6d18af4 point to MinIO instance instead of through proxy 2022-01-30 21:25:56 +01:00
Martin McKeaveney 7c4d7e0e8f nginx prod config, security headers and content-security policies 2022-01-30 21:11:59 +01:00
Maurits Lourens 2577a6631a run prettier + lint over typescript files as well 2022-01-20 09:17:08 +01:00
Martin McKeaveney 93caf5bede Merge branch 'master' of github.com:Budibase/budibase into feature/envoy-nginx 2022-01-13 18:53:34 +01:00
Martin McKeaveney 6721b234ac nginx dev setup 2022-01-11 20:33:22 +01:00
mike12345567 1327394784 Switching out @budibase/auth to @budibase/backend-core. 2022-01-10 19:33:00 +00:00
Rory Powell 6118b7ba36 Fix oracle password config 2021-12-03 10:52:03 +00:00
Rory Powell 36218ccea9 Fix oracle password config 2021-12-03 10:48:47 +00:00
Rory Powell 301938eaa0 Merge pull request #3555 from Budibase/oracle/returning
Improve returning logic for oracle
2021-11-26 18:54:19 +00:00
Rory Powell 115fc08aa4 Improve returning logic for oracle 2021-11-26 15:02:03 +00:00
mike12345567 4697b21ef1 Updating docs for tables, adding a swagger generation tool and a README describing how to use it. 2021-11-26 14:13:53 +00:00
mike12345567 a99bf1bba3 Updating scripts to allow opening in the browser when generated. 2021-11-25 18:13:14 +00:00
mike12345567 5b2e6bddf8 Updating docs. 2021-11-25 17:44:47 +00:00
mike12345567 88f31bcf34 Basic work for generating. 2021-11-25 16:13:19 +00:00
Rory Powell 1a0c66beb6 Merge branch 'develop' into oracle-datasource 2021-11-23 09:27:55 +00:00
mike12345567 0335b1cee3 Fixing issue from review, values weren't wiped when delete modals closed, also adding a fix for threading to disable it in development as node-ts was causing memory leaks on low memory systems (doesn't apply to production built version). 2021-11-22 17:42:41 +00:00
Rory Powell f94a0eadbe Merge branch 'develop' into oracle-datasource 2021-11-22 10:47:48 +00:00
mike12345567 7170c00a4f Fix for cypress test issues, when metadata is updated rapidly it could get into a bad state - this should resolve it. 2021-11-16 20:56:24 +00:00
mike12345567 aaff5260d3 Adding a script to be able to debug backend like cypress runs, without needing to build everytime - fixing an issue that appears to only occur occasionally in the cypress environment. 2021-11-16 17:40:31 +00:00
Rory Powell ef2697a82d Add oracle instant client installation scripts 2021-11-09 17:10:26 +00:00
Rory Powell cb661eab34 Add oracledb package, readme, and config 2021-11-05 13:56:54 +00:00
mike12345567 9c933b629f Linting and updating SQL Server schema generation to include auto column and primary key recognition. 2021-11-05 12:33:48 +00:00
Rory Powell b5b191361d Add stub integration file and docker compose config 2021-11-04 14:54:35 +00:00
mike12345567 09d83dea39 Adding docker compose for ms-sql with products, tasks table setup. 2021-10-27 17:32:02 +01:00
Rory Powell ad61f2af3b Prevent non builder from accessing dev apps 2021-10-25 16:59:09 +01:00