Commit Graph

55 Commits

Author SHA1 Message Date
Adria Navarro 9cd7ef7827 Typing redis 2023-09-19 14:32:05 +02:00
Adria Navarro 10fe86e30d Type 2023-09-18 12:44:58 +02:00
mike12345567 c13530ef32 Stop attempt at bundling ioredis-mock, only need to check at runtime. 2023-06-02 13:59:33 +01:00
mike12345567 ddeb25b68d Updating mock redis to only import if it is needed. 2023-06-02 11:52:50 +01:00
Michael Drury a3f9759a47 Reverting ioredis-mock removal for test cases - will come back to this in future as it currently breaks pro tests. 2023-06-01 00:02:55 +01:00
mike12345567 fba3ec987e Merge branch 'develop' of github.com:Budibase/budibase into fix/datasource-info-endpoint 2023-05-31 17:37:17 +01:00
mike12345567 249625ad1a Updating backend tests to actually utilise the redis container which is spun up as part of the tests, doing away with the somewhat problematic ioredis-mock which was breaking some tests due to the addition of the changes for multi-dev collab. 2023-05-31 17:18:01 +01:00
Andrew Kingston 8a2d4b7052 Merge branch 'develop' of github.com:Budibase/budibase into websocket-redis-integration 2023-05-31 16:28:14 +01:00
Andrew Kingston e3b5d711df Disable redis select command in tests 2023-05-31 16:27:49 +01:00
Andrew Kingston ffbbf04e86 Lint 2023-05-31 16:08:35 +01:00
Andrew Kingston 05e34076f7 Fully rewrite websocket redis integration to use individual keys per session, enable TTLs on sesisons, prune sessions when users connect and add a heartbeat to sockets 2023-05-31 15:13:22 +01:00
Andrew Kingston d926650232 Add back in functionality to select different redis DB's per client 2023-05-31 10:52:39 +01:00
Michael Drury 99607ca06e Reverting try twice change to writethrough. 2023-05-30 20:45:10 +01:00
mike12345567 7c7bd4d5cb Fix for debugging with webstorm the old way (if desired), updating the builder middleware to be more multi-dev capable, ignoring 409s when attempting to update the last updated at for apps (if multiple devs hit at same time, only use one) also updating writethrough cache to retry once, with the extended TTL on locks, plus the multi-dev collab it can take a minute to update usage quota doc when a lot of updates occur at once. 2023-05-30 17:41:20 +01:00
Andrew Kingston 5e480a1527 Use socker.io redis adapter to broadcast events to all server instances 2023-05-25 08:48:56 +01:00
Rory Powell 595db7d346 Merge pull request #10495 from Budibase/fix/custom-locks
Fix usage of custom locks
2023-05-05 15:21:58 +01:00
Rory Powell 1ba5eda149 Fix usage of custom locks 2023-05-05 14:42:21 +01:00
Martin McKeaveney 50de32a8a0 Merge pull request #10452 from Budibase/feature/clustering-support
attaching clustered mode to an env var
2023-05-04 14:25:23 +01:00
Martin McKeaveney 167dfd39d7 fix ts build 2023-05-04 13:32:17 +01:00
Martin McKeaveney e75e91e248 control all redis clustering through env var 2023-04-28 19:53:15 +01:00
Martin McKeaveney f73af919a7 attaching clustered mode to an env var 2023-04-28 19:09:22 +01:00
Rory Powell ec06f13aa6 Per user pricing (#10378)
* Update pro version to 2.4.44-alpha.9 (#10231)

Co-authored-by: Budibase Staging Release Bot <>

* Track installation and unique tenant id on licence activate (#10146)

* changes and exports

* removing the extend

* Lint + tidy

* Update account.ts

---------

Co-authored-by: Rory Powell <rory.codes@gmail.com>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>

* Type updates for loading new plans (#10245)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS` (#10247)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete + migration (#10250)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Lint

* Types and structures for user subscription quantity sync (#10280)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Prevent old installs from activating, track install info via get license request instead of on activation.

* Add usesInvoicing to PurchasedPlan

* Add min/max users to PurchasedPlan

* Additional test structures for generating a license, remove maxUsers from PurchasedPlan - this is already present in the license quotas

* Stripe integration for monthly prorations on annual plans

* Integrate annual prorations with test clocks

* Updated types, test utils and date processing for licensing (#10346)

* Add new quota for max users on free plan

* Split available vs purchased plan & price type definitions. Update usages of available prices and plans

* Type fixes

* Add types for minimums

* New `PlanModel` type for `PER_USER` and `DAY_PASS`

* Add loadEnvFiles to lerna config for run command to prevent local test failures

* Fix types in license test structure

* Add quotas integration to user create / delete

* Always sync user count from view total_rows value for accuracy

* Add migration to sync users

* Add syncUsers.spec.ts

* Prevent old installs from activating, track install info via get license request instead of on activation.

* Add usesInvoicing to PurchasedPlan

* Add min/max users to PurchasedPlan

* Additional test structures for generating a license, remove maxUsers from PurchasedPlan - this is already present in the license quotas

* Stripe integration for monthly prorations on annual plans

* Integrate annual prorations with test clocks

* Updated types, test utils and date processing

* Lint

* Pricing/billing page (#10353)

* bbui updates for billing page

* Require all PlanTypes in PlanMinimums for compile time safety

* fix test package utils

* Incoming user limits warnings (#10379)

* incoming user limits warning

* fix inlinealert button

* add corretc button link and text to user alert

* pr comments

* simplify limit check

* Types and test updates for subscription quantity changes in account-portal (#10372)

* Add chance extensions for `arrayOf`. Update events spies with license events

* Add generics to doInTenant response

* Update account structure with quota usage

* User count limits (#10385)

* incoming user limits warning

* fix inlinealert button

* add corretc button link and text to user alert

* pr comments

* simplify limit check

* user limit messaging on add users modal

* user limit messaging on import users modal

* update licensing store to be more generic

* some styling updates

* remove console log

* Store tweaks

* Add startDate to Quota type

---------

Co-authored-by: Rory Powell <rory.codes@gmail.com>

* Lint

* Support custom lock options

* Reactivity fixes for add user modals

* Update ethereal email creds

* Add warn for getting invite from code error

* Extract disabling user import condition

* Handling unlimited users in modals logic and adding start date processing to store

* Lint

* Integration testing fixes (#10389)

* lint

---------

Co-authored-by: Mateus Badan de Pieri <mateuspieri@gmail.com>
Co-authored-by: mike12345567 <me@michaeldrury.co.uk>
Co-authored-by: Peter Clement <PClmnt@users.noreply.github.com>
2023-04-24 09:31:48 +01:00
mike12345567 cf5316ec8d General fixes for open handles, attempting to find and close all issues in server which are stopping shutdown of Jest suite. 2023-03-27 19:38:49 +01:00
adrinr 84d450a931 Renames 2023-03-07 12:47:27 +01:00
adrinr c1462a7c9c Clean code 2023-02-28 14:54:43 +01:00
adrinr 66217d6b08 Return redlock execution info 2023-02-28 12:52:43 +01:00
adrinr 24161d12bf Type locks 2023-02-28 12:47:28 +01:00
mike12345567 d6b0b7610a Fixing circular dependencies which were blocking tests. 2023-02-24 17:31:58 +00:00
Rory Powell 6a88cfc32d Update locks error logging (#9768)
* Fix intermittent backend-core migration test failure

* Update lock logging
2023-02-22 08:32:03 +00:00
Rory Powell 940de8b6a0 Run CI steps in parallel (#9760)
* Parallel CI

* Add build to integration test

* Add checkout to top of each run

* Revert branch update for ci job

* Experiment with --runInBand for CI

* Fix intermittent backend-core migration test failure

* Fix hanging worker redis connection

* Update naming from reset to newTenant
2023-02-21 17:13:24 +00:00
Rory Powell eb5aa8786d Merge pull request #9670 from Budibase/budi-6559-enable-higher-concurrency-and-resiliency
Enable higher concurrency and resiliency in worker tests
2023-02-13 14:31:14 +00:00
Rory Powell 07e5598538 Enable use of redis container in worker tests 2023-02-13 12:09:16 +00:00
Rory Powell ae9979929a Enable higher concurrency and resiliency in worker tests 2023-02-13 11:57:30 +00:00
Rory Powell fba7ef26ad Replace ts-jest with swc/jest (#9289)
- Add swc dependencies
- Add transform change to jest.config.ts
- Replace `export =` with `export default` in src code to enable code coverage to work with swc transformer
- Restructure backend-core errors package to allow for exporting error classes with ESM syntax
- Update backend-core to no longer use `export =`, export individual packages instead of replacing with `export default` for backwards compatibility
- Update event publishers to use `export default` - this was required for the `jest.spyOn` usage inside backend-core common mocks
- Restructure some jest.mock usages where declaring the jest.fn as a variable outside of the package mock threw an error
2023-01-11 09:37:37 +00:00
Martin McKeaveney 7fb565e9f4 lock changes (#9047) 2022-12-15 12:45:53 +00:00
mike12345567 d3c4c99e16 Complete conversion of backend-core to Typescript. 2022-11-24 18:48:51 +00:00
Rory Powell 56456f3f6d Startup fixes 2022-10-25 13:37:26 +01:00
Rory Powell 281eb151e2 Merge pull request #8164 from Budibase/feature/quota-emails
Approaching and Exceeded Usage Limit Notifications
2022-10-12 12:31:17 +01:00
Martin McKeaveney 536b7238e9 redis fix for new instances, removing relationship options from google sheets integration 2022-10-10 11:00:45 +01:00
Rory Powell 84685d3340 Add locking framework 2022-10-10 08:21:17 +01:00
mike12345567 197d195cb4 Switching from scan for app locks to mget - which is a fast O(N) operation that only retrieves what we need. 2022-10-07 18:05:18 +01:00
mike12345567 53e9f79c65 Adding explanation of the difference between Database and SelectableDatabase in Redis. 2022-06-24 14:42:15 +01:00
mike12345567 b4bed6c0ce Updating redis to use typescript and adding the option of a writethrough cache which can be used, by passing a DB and a value to be written + a delay for writes. 2022-06-23 20:22:51 +01:00
Rory Powell 361ffd224f Merge branch 'release' into feature/event-backfill 2022-06-06 09:50:06 +01:00
Rory Powell 86d094dda4 Migration locks and add optional preventRetry option 2022-06-01 17:52:41 +01:00
Rory Powell 081db8423e Fix key eviction 2022-06-01 15:31:27 +01:00
Rory Powell 7ebfaf8a31 Failover protection with event cache 2022-05-31 21:04:41 +01:00
mike12345567 fa30457f2b Fixing issue with server not shutting down correctly when an error occurs, making sure that everything clears up gracefully. 2022-05-30 21:22:06 +01:00
mike12345567 76f19def26 Moving the withCache function to backend-core so that it can be shared, as well as adding a cache busting function which can be used when one of the checklist configs changes. 2022-05-23 15:03:52 +01:00
Martin McKeaveney 6d3aa6a806 withCache wrapper for working with redis 2022-05-23 00:09:03 +01:00