Commit Graph

283 Commits

Author SHA1 Message Date
mike12345567 4f7e56fed9 Making build script into a module, so that it can be used in a specialised server build which also builds the thread scripts as required for worker-farm. 2023-04-27 19:00:01 +01:00
mike12345567 c552913737 Fixing defaults in MS-SQL as well. 2023-04-14 18:10:10 +01:00
mike12345567 09ccac12ae Fix for default values with NOT NULL requirements being intepreted as requiring values in the frontend, presence check is now ignored if a default or auto value is found. 2023-04-14 17:56:17 +01:00
Rory Powell 77ffb8d86d Run integration suite in CI again / auto detect tenancy / refactors (#10209)
* qa-core-ci-fixes

* global setup and teardown wip

* Updates to logs and setup

* Remove date and console mocking

* Update CI to spin up minimal dev env

* Update readme

* Fix scopeBackend.sh

* Ensure docker services are initialised before starting worker

* Lint

* Fix admin user being created on startup (#10219)

* use regular bootstrap and build

* Lint

* Temp: re-use global setup to get around app limit in QA
2023-04-05 15:33:56 +01:00
Rory Powell 976b3a55ca Update logging to support dd trace attributes (#10086)
* Update logging middleware to integrate with pino for console logging

* Remove elastic apm references, use updated core middlewares

* Remove redundant LOG_LEVEL definitions

* Remove no longer needed jest logging overrides

* lint

* Backwards compat between console log helpers and pino

* Configurable DISABLE_HTTP_LOGGING

* Don't log 4xx as errors

* Remove redundant ENABLE_4XX_HTTP_LOGGING

* Cleanup migrations and event logging

* Improve bb-alert logging

* Add DISABLE_HTTP_LOGGING to helm chart

* Add ops endpoints for testing

* Disable http logging in dev

* Backwards compatible tracing implementation

* Naming update on http logging env var

* lint

* Update packages/backend-core/src/environment.ts

Co-authored-by: Adria Navarro <adria@revityapp.com>

* Merge

* Lint

* Fix console.warn failing mock by replacing with alerts mock instead

* Lint

---------

Co-authored-by: Adria Navarro <adria@revityapp.com>
2023-04-04 15:08:46 +01:00
Rory Powell 79bec7d86d Add force exit, -e flag and streaming to tests (#10151)
* Add force exit, -e flag and streaming to tests

* Temp remove script usage from worker tests

* Revert "Temp remove script usage from worker tests"

This reverts commit 5e0ed2e92d.

* Always fail test

* Ignore server test package from coverage report

* Unscope server only

* Run all server tests again

* Update backend core test script to fail when any command fails and force exit

* Don't try to require client in test

* Port memory limit fix from develop

* Remove always failing test and commented out block for client require

* Fixing some issues with test cases.

* Revert assertions in internalSearch.spec.js

---------

Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
2023-03-31 10:32:13 +01:00
Rory Powell 9d6d781244 Use `--runInBand` in CI only (#9771)
* Use  in CI only

* Use scripts instead of ternary to pick up correct bash syntax
2023-02-22 13:04:16 +00:00
mike12345567 3f7f66257d Merge branch 'master' of github.com:Budibase/budibase into develop 2023-02-14 17:57:23 +00:00
mike12345567 488b542ac1 Fix for #9654 - the error checking was looking at the through fields flipped, which caused many to many relationships to throw an error. Also I've updated the dev compose file on master so that switching between develop and master is smoother. 2023-02-14 14:38:55 +00:00
mike12345567 3b43943f1f Some quick scripts for loading a lot of rows and apps. 2023-02-03 19:29:28 +00:00
Rory Powell 905de31cd6 Merge pull request #9525 from Budibase/chore/default-local-dev-accounts-disabled
Set DISABLE_ACCOUNT_PORTAL=1 by default in local dev
2023-02-03 09:54:21 +00:00
Rory Powell f28b1dc79a Set DISABLE_ACCOUNT_PORTAL=1 by default in local dev 2023-02-02 16:38:23 +00:00
Dean 0eed617866 Updated default feature flags to include onboarding by default. Added tenant overrides to allow exclusion from global features 2023-02-01 14:12:43 +00:00
mike12345567 95e8371b1d Adding encryption key to manage of server. 2023-01-16 18:51:48 +00:00
Martin McKeaveney 1dcd3445ce adding host.docker.internal mapping to PROXY_ADDRESS 2022-12-21 10:58:39 +00:00
Rory Powell b635614f77 Allow proxy upstream URLs to be configured at runtime (#7413)
* allow proxy upstream urls to be configured at runtime

* make dynamic upstream urls in proxy the default behaviour

* reorder env directives

* Use appVersion for proxy image

* Update couch upstream for proxy

* proxy ci updates - WIP

* allowing custom resolvers in proxy service deployment

* remove generateProxyConf script for prod and remove from builds

* remove generateProxyConfig script

* removing nginx dev conf

* delete generated nginx conf

Co-authored-by: Thomas Weiß <panos@unbunt.org>
Co-authored-by: Martin McKeaveney <martinmckeaveney@gmail.com>
2022-12-20 16:28:13 +00:00
mike12345567 2a51091ffb Adding some null controls. 2022-12-02 16:17:56 +00:00
mike12345567 32488bee88 Adding broken SQL configuration. 2022-12-02 10:50:52 +00:00
mike12345567 02ede5d067 Finally removing all usages of backend-core/ type imports from server, including some further typescript conversions. 2022-11-22 19:49:59 +00:00
Rory Powell 1612044dc1 Merge branch 'develop' into subdomain-tenancy 2022-11-15 16:18:59 +00:00
Rory Powell 9973f2e161 Bump jest and ts-jest to be consistent across backend packages to fix CI issue, test config updates 2022-11-14 22:55:47 +00:00
Rory Powell 5da699e27c Fail pro scripts on error, unify TS versions, update test and coverage configs, use pro source mappings 2022-11-14 13:29:46 +00:00
Rory Powell 6783fd713e Update tenancy detection to honour any subdomain pattern according to platform url 2022-11-09 16:35:16 +00:00
Rory Powell 5d973de503 Add TENANT_FEATURE_FLAGS to default dev environment 2022-09-23 13:48:19 +01:00
Andrew Kingston 5bcb4c5393 Move restart of proxy container to after the compose stack has been brought up to ensure container exists 2022-09-16 09:25:14 +01:00
Andrew Kingston 206714869d Always restart the proxy service when spinning up the dev stack to account for nginx conf changes 2022-09-15 16:31:29 +01:00
Rory Powell a551532e73 Navigation updates, cookie constant, update link-dependencies and localdomain scripts 2022-09-14 10:53:14 +01:00
Andrew Kingston 6093da79d6 Add hot reloading for component plugins via watched plugin directory 2022-08-15 22:23:45 +01:00
Andrew Kingston 467cc27f47 Add env vars to automatically create initial admin user on first run 2022-06-30 11:28:52 +01:00
mike12345567 e32de3b850 Updating install script to work from any directory. 2022-06-21 20:01:50 +01:00
mike12345567 9ad9860a9c Fixing cleanup script, adding ARM support to dockerfile, as well as adding ARM client. 2022-06-21 17:56:55 +01:00
Rory Powell 12aa9fa1ce Dev improvements - remove need to rebuilds of backend-core/dist and enable debugging when running tests and backend services, use source directly using typescript and jest path mappings 2022-06-10 22:57:12 +01:00
Rory Powell 0cf777a0cd Build fixes 2022-06-09 12:33:10 +01:00
Rory Powell a4eef3daab Add awareness of deployment environment 2022-06-01 14:10:00 +01:00
Rory Powell a9ab8b2629 Lots of failure handling and caching 2022-05-30 21:46:08 +01:00
Rory Powell 28b48a88fe Event identification 2022-05-23 22:14:44 +01:00
Rory Powell 70fff5c7b9 Merge branch 'develop' into feature/posthog-v2 2022-05-23 09:04:00 +01:00
mike12345567 c528896e2d Finishing off UI, expanding settings menus from 80ch to 90ch as they were quite restricted and made the usage UI a bit difficult to view. Added a DISABLE_DEVELOPER_LICENSE env variable for testing. 2022-05-17 17:44:40 +01:00
Rory Powell 1caf4b1965 Merge branch 'develop' into feature/posthog-v2 2022-04-27 16:32:00 +01:00
Martin McKeaveney 9ca4dacbaa merge with master 2022-04-26 10:21:45 +01:00
Michael Drury 94419024a4 Merge pull request #5510 from Budibase/fix/pouch-leak
PouchDB memory leak protection
2022-04-22 15:32:51 +01:00
mike12345567 9899781fc5 Some updates, making sure databases will always close directly using finally checks around the actual tasks, updating how replication works to have a close statement (to make sure it is controlled correctly) and then updating to PouchDB 7.3.0 for one of the memory leak fixes. 2022-04-20 17:33:42 +01:00
mike12345567 192fb1307e Initial version of memory leak protection, making sure that PouchDB databases are closed correctly after use, using a combination of closures wrapping DB gets (this replaces the getDB, leaving only a dangerousGetDB function which can be used in very very specific scenarios) and then closing the DB as part of CLS hooked functions finishing. Also moving the GlobalDB init to the tenancy middleware as this is used everywhere in the worker/app services - means that not all getGlobalDB calls require an async closure around them. 2022-04-19 19:42:52 +01:00
Rory Powell 41144db055 Remove remaining direct usage of CouchDB. Add common pouch config to backend-core 2022-04-19 16:45:27 +01:00
mike12345567 5180603374 Fix for #5411 - trimming the various binding components for SQL in queries. Also moving SQL interpolation out into a Typescript - prep for further unit testing/improvements. 2022-04-14 13:06:22 +01:00
Rory Powell 33719fcb09 Merge branch 'feature/licensing' into feature/posthog-v2 2022-04-12 17:12:46 +01:00
Rory Powell 03ba6d914f Merge branch 'develop' into feature/licensing 2022-04-12 16:18:41 +01:00
Rory Powell 5ab9f1a9c5 Table CRUD events + add timekeeper for tests 2022-04-11 14:39:31 +01:00
mike12345567 39182cfbc0 Fixing issue #5817 - making sure that date strings are correctly parsed into the bindings. 2022-04-01 17:13:28 +01:00
Rory Powell 15b676ce1c Remove remaining direct usage of CouchDB. Add common pouch config to backend-core 2022-03-29 16:03:44 +01:00
Rory Powell 75d5c0c0b9 Merge branch 'feature/licensing' into feature/posthog-v2 2022-03-28 09:47:34 +01:00
Rory Powell 69b2b38aea Merge branch 'master' into merge-master 2022-03-25 13:10:37 +00:00
Martin McKeaveney 14c75b2907 update always to unless-stopped 2022-03-24 17:35:35 +00:00
mike12345567 d64df5fd78 Adding a fix for #4996 - checks if WSL is in use when configuring nginx. 2022-03-23 15:31:08 +00:00
Rory Powell 715d42d3e6 Merge branch 'develop' into feature/licensing 2022-03-21 17:13:16 +00:00
Martin McKeaveney 45f158dcc5 Merge pull request #4587 from CommanderRoot/rm-deprecated-substr
Replace deprecated String.prototype.substr()
2022-03-10 01:57:57 +01:00
mike12345567 7f9334ae5e Making sure that tables are created within the correct schema for MS-SQL. 2022-03-04 15:50:19 +00:00
mike12345567 a4ed8fe5e8 Adding the ability to support schemas in MS-SQL. 2022-03-04 15:12:07 +00:00
Rory Powell a81041bc40 Add shared licensing cache 2022-03-03 13:37:04 +00:00
Martin McKeaveney b28bffbb15 Merge pull request #4631 from Budibase/ci/deploy-proxy
removing need for handlebars in nginx script, automating proxy servic…
2022-03-03 09:06:49 +00:00
mike12345567 1b45a9190d Fixes for SQL issues raised by Marty. 2022-03-02 17:40:50 +00:00
mike12345567 8b6a9b7273 Merge branch 'develop' of github.com:Budibase/budibase into feature/budibase-api 2022-02-28 18:43:45 +00:00
mike12345567 c1a1c02e73 Updating validators, make sure everything that is optional is really optional. 2022-02-25 11:39:46 +00:00
Rory Powell 83d8c70419 Change ports 2022-02-23 07:00:57 +00:00
Martin McKeaveney 1cbca0578a removing need for handlebars in nginx script, automating proxy service deployment 2022-02-22 13:24:21 +01:00
Rory Powell b9306b5a7a Prevent local port conflicts between account portal and cypress 2022-02-22 11:20:58 +00:00
Tobias Speicher 491bcb86ed Replace deprecated String.prototype.substr()
String.prototype.substr() is deprecated (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/substr) so we replace it with slice() which works similarily but isn't deprecated.
Signed-off-by: Tobias Speicher <rootcommander@gmail.com>
2022-02-20 15:28:39 +01:00
Rory Powell 02c0c49594 Deprovision and local couch db url fix 2022-02-18 17:36:23 +00:00
Martin McKeaveney a7a82dc679 adding k8s resolver and updating watchtower variable 2022-02-16 15:33:57 +01:00
Martin McKeaveney 97bf3e68dd adding watchtower service 2022-02-16 14:29:08 +01:00
Martin McKeaveney a85f667dbe templatize nginx configuration into one prod configuration and add NPM scripts to manage 2022-02-16 14:12:53 +01:00
Martin McKeaveney a495050871 merge with develop 2022-02-10 11:39:15 +01:00
mike12345567 d13985af3f Fix for #3758 - updating the datetime cell renderer and form component to be capable of display time only formats, as well as adding an option to only select the time component for a date time form field. 2022-02-08 16:15:08 +00:00
mike12345567 f61e15594b Fix for #3928 - error invalid use of FETCH was based on a lack of sorting on MS-SQL platforms, this enforces a sort of some type no matter what. 2022-02-04 16:17:36 +00:00
mike12345567 168a126e2a Updating the SQL core to allow it to handle multiple relationships between the same two tables. 2022-02-02 18:15:17 +00:00
Martin McKeaveney c11e26cb2e merge 2022-02-01 11:22:14 +01:00
Martin McKeaveney 7fb9b33f6c remove more references to envoy 2022-02-01 11:02:37 +01:00
Martin McKeaveney b521721642 fixing cypress setup 2022-01-31 16:35:23 +01:00
Martin McKeaveney cca6d18af4 point to MinIO instance instead of through proxy 2022-01-30 21:25:56 +01:00
Martin McKeaveney 7c4d7e0e8f nginx prod config, security headers and content-security policies 2022-01-30 21:11:59 +01:00
Maurits Lourens 2577a6631a run prettier + lint over typescript files as well 2022-01-20 09:17:08 +01:00
Martin McKeaveney 93caf5bede Merge branch 'master' of github.com:Budibase/budibase into feature/envoy-nginx 2022-01-13 18:53:34 +01:00
Martin McKeaveney 6721b234ac nginx dev setup 2022-01-11 20:33:22 +01:00
mike12345567 1327394784 Switching out @budibase/auth to @budibase/backend-core. 2022-01-10 19:33:00 +00:00
Rory Powell 6118b7ba36 Fix oracle password config 2021-12-03 10:52:03 +00:00
Rory Powell 36218ccea9 Fix oracle password config 2021-12-03 10:48:47 +00:00
Rory Powell 301938eaa0 Merge pull request #3555 from Budibase/oracle/returning
Improve returning logic for oracle
2021-11-26 18:54:19 +00:00
Rory Powell 115fc08aa4 Improve returning logic for oracle 2021-11-26 15:02:03 +00:00
mike12345567 4697b21ef1 Updating docs for tables, adding a swagger generation tool and a README describing how to use it. 2021-11-26 14:13:53 +00:00
mike12345567 a99bf1bba3 Updating scripts to allow opening in the browser when generated. 2021-11-25 18:13:14 +00:00
mike12345567 5b2e6bddf8 Updating docs. 2021-11-25 17:44:47 +00:00
mike12345567 88f31bcf34 Basic work for generating. 2021-11-25 16:13:19 +00:00
Rory Powell 1a0c66beb6 Merge branch 'develop' into oracle-datasource 2021-11-23 09:27:55 +00:00
mike12345567 0335b1cee3 Fixing issue from review, values weren't wiped when delete modals closed, also adding a fix for threading to disable it in development as node-ts was causing memory leaks on low memory systems (doesn't apply to production built version). 2021-11-22 17:42:41 +00:00
Rory Powell f94a0eadbe Merge branch 'develop' into oracle-datasource 2021-11-22 10:47:48 +00:00
mike12345567 7170c00a4f Fix for cypress test issues, when metadata is updated rapidly it could get into a bad state - this should resolve it. 2021-11-16 20:56:24 +00:00
mike12345567 aaff5260d3 Adding a script to be able to debug backend like cypress runs, without needing to build everytime - fixing an issue that appears to only occur occasionally in the cypress environment. 2021-11-16 17:40:31 +00:00
Rory Powell ef2697a82d Add oracle instant client installation scripts 2021-11-09 17:10:26 +00:00
Rory Powell cb661eab34 Add oracledb package, readme, and config 2021-11-05 13:56:54 +00:00
mike12345567 9c933b629f Linting and updating SQL Server schema generation to include auto column and primary key recognition. 2021-11-05 12:33:48 +00:00
Rory Powell b5b191361d Add stub integration file and docker compose config 2021-11-04 14:54:35 +00:00
mike12345567 09d83dea39 Adding docker compose for ms-sql with products, tasks table setup. 2021-10-27 17:32:02 +01:00
Rory Powell ad61f2af3b Prevent non builder from accessing dev apps 2021-10-25 16:59:09 +01:00
Rory Powell 5ffed2824e Introduce dev modes, update contributor guide 2021-10-08 13:13:16 +01:00
Martin McKeaveney bfb7766333 Merge pull request #2857 from Budibase/fix/2792
Allow SQL many to many relationships to use arbitrarily named columns in junction table
2021-10-04 12:42:56 +01:00
Rory Powell d63186ff8f Fix email link from internal bb onboarding. Prevent account holder from being deleted 2021-10-04 11:30:59 +01:00
mike12345567 80e7671444 Fix for many to many relationships where the union table has arbituarily named foreign key constraint columns, an example has been provided in the scripts directory. 2021-10-01 17:16:43 +01:00
mike12345567 a0f69251c8 Adding the ability to run the monorepo in cloud mode without the connected account portal (login, org setting and admin creation are all available again) as well as re-working how the redirects work to massively reduce the chance of cycles which I was experiencing constantly. 2021-09-29 16:55:59 +01:00
mike12345567 d282cd030f Some fixes after testing and adding some basic work to the frontend. 2021-09-28 18:05:52 +01:00
mike12345567 32bdc4a991 Minimal fix for issue involving JSON views and newlines with postgres, this fix couldn't be more over-arching as it risked breaking new lines across the board. Have included a script for setting up the test scenario as well. This fixes issue #2612. 2021-09-24 18:10:30 +01:00
mike12345567 67212e5c5b Fixes issue #2616 - this is a slightly complex fix and handles a few other issues with mysql (around returning on creation of a row and relationships) - a new mechanism is now used for pagination and limiting which makes sure the limits are applied to the outer table rather than the combination of the outer and the joined. 2021-09-23 16:35:09 +01:00
mike12345567 e92b0e0c94 Fixing issue #2673 - discovered issues with some column types not being worked out to be auto-columns in postgres. 2021-09-22 17:46:54 +01:00
Rory Powell 00e2b50f26 fix tests 2021-09-20 12:58:58 +01:00
Rory Powell 62e93d2302 Add self host enable/disable scripts 2021-09-14 12:21:16 +01:00
Martin McKeaveney f4232bd7d7 couchDB auth options 2021-08-17 14:11:03 +01:00
mike12345567 a8eb067de3 Merge branch 'fix/apps-2195-2282-2283' of github.com:Budibase/budibase into develop 2021-08-06 17:14:31 +01:00
mike12345567 be48677490 Fixes for a lot of issues generated by the use of the pouchdb allDbs call, which is not designed for multi-client DB setups like ours, using CouchDB method instead. 2021-08-06 16:38:07 +01:00
mike12345567 46ea3f3e9e Revert "Merge pull request #2253 from Budibase/revert-2076-feature/multi-tenants"
This reverts commit 4834b765be, reversing
changes made to 2456e69483.
2021-08-05 09:59:08 +01:00
Martin McKeaveney df927e3feb merge with master 2021-08-04 13:20:11 +01:00
Michael Drury ec26f6ab5e Merge branch 'develop' into feature/multi-tenants 2021-08-04 10:07:31 +01:00
Michael Drury b86a6fddc9 Revert "Multi-tenancy/organisations" 2021-08-04 10:02:24 +01:00
Rory Powell 648661877a Add watchtower hooks 2021-07-29 20:32:18 +01:00
mike12345567 615053f201 Fixing some issues with org not being correctly detected. 2021-07-27 15:35:26 +01:00
mike12345567 219962cb8a Adding ability to disable/enable multi-tenancy flags easily. 2021-07-26 16:17:56 +01:00
mike12345567 4606248ffd Fixing up everything found when testing with Cypress. 2021-07-26 13:19:47 +01:00
mike12345567 4d0faa38e3 Changing how apps are retrieved to be more efficient. 2021-07-23 20:01:53 +01:00
mike12345567 1d6a350311 Changing how flags are handled. 2021-07-22 16:36:21 +01:00
mike12345567 7aca56f019 Adding float type to MySQL. 2021-07-12 11:10:11 +01:00
mike12345567 184df6c685 Fixing some issues discovered with the new SQL connectors were the input type would always be string (causing some SQL types to break) - parsing these before input to attempt to fix this problem - issue referenced in #1943. 2021-07-12 10:51:30 +01:00
mike12345567 8550d00751 Giving the system the ability to make auto increment columns auto-columns in postgres or mysql. 2021-07-05 18:11:23 +01:00
mike12345567 98b7bff678 Implementing all return possibilities, now to implement creation. 2021-06-29 17:42:46 +01:00
mike12345567 10cf55e273 Creating a many-to-many database scenario and seeding with data. 2021-06-28 12:36:05 +01:00
mike12345567 f0e7b9b25e Speeding up server tests and removing the ability for it to port collide. 2021-06-25 16:14:06 +01:00
mike12345567 b3f3826aa6 Fixing an issue I found with postgres schema generator, it was applying all primary keys to all tables. 2021-06-23 12:44:54 +01:00
mike12345567 9cc93b62a7 Adding script for creating basic postgres setup. 2021-06-18 15:03:22 +01:00
mike12345567 21b591954f Adding a mySQL plus integration, as well as fixing some issues with running queries directly. 2021-06-18 12:29:25 +01:00
mike12345567 033258b495 Updating export template script to remove users as well. 2021-06-08 18:09:09 +01:00
Martin McKeaveney 0f2bcf581d revert functionality working 2021-05-16 21:25:37 +01:00
mike12345567 8e1f18e456 Changing INTERNAL_KEY to INTERNAL_API_KEY. 2021-05-11 15:23:03 +01:00
mike12345567 7736869148 Some updates, working towards supporting automation send smtp email also removing the styling template, adding to base. 2021-05-11 12:02:29 +01:00
mike12345567 1cf778845a Fixing issues with Redis/Bull and the integration with new redis module. 2021-05-04 18:13:44 +01:00
Martin McKeaveney 8ecb70a8f7 merge with next 2021-05-04 11:40:59 +01:00
Andrew Kingston 797e8a8b67 Add explicit prettier options 2021-05-04 11:32:22 +01:00
Martin McKeaveney 72ed644803 merge 2021-05-04 09:55:14 +01:00
Keviin Åberg Kultalahti 0afbf1649e lint:fix 2021-05-03 09:31:09 +02:00
Martin McKeaveney 6d32d5f8b1 tidy up 2021-04-30 15:44:37 +01:00
Martin McKeaveney b876e91ae6 merge with next 2021-04-29 11:45:47 +01:00
mike12345567 0c34f1a108 Making sure volumes are removed when nuking. 2021-04-08 18:18:53 +01:00
mike12345567 15900efc35 Some changes after trying system for first time. 2021-04-08 17:39:46 +01:00
mike12345567 16b5b19027 Updating to make sure the envoy generated dev file is very obviously not to be edited. 2021-04-07 17:52:22 +01:00