diff --git a/LICENSE b/LICENSE index a017209adf..cbb55109f4 100644 --- a/LICENSE +++ b/LICENSE @@ -1,7 +1,9 @@ -Copyright 2019-2021, Budibase Ltd. +Copyright 2019-2023, Budibase Ltd. Each Budibase package has its own license, please check the license file in each package. You can consider Budibase to be GPLv3 licensed overall. The apps that you build with Budibase do not package any GPLv3 licensed code, thus do not fall under those restrictions. + +Budibase ships with Structured Query Server, by The Neighbourhoodie Software GmbH. This license for this can be found at ./SQS_LICENSE diff --git a/SQS_LICENSE b/SQS_LICENSE new file mode 100644 index 0000000000..0315ee9527 --- /dev/null +++ b/SQS_LICENSE @@ -0,0 +1,31 @@ +FORM OF CUSTOMER LICENCE + +Budibase hereby grants the Customer a worldwide, royalty free, non-exclusive, +perpetual (for the lifetime of the intellectual property rights contained in the Product) +right and title to utilise the binary code of the The Neighbourhoodie Software GmbH +Structured Query Server software product (Product) for its own internal business +purposes (the Purpose) only (the Licence). The Product has the function of bringing a +CouchDB database (NoSQL database) into an SQL database form (SQLite) and thereby +making it usable for complex queries - which originally could only be displayed in an +SQL database. By indexing in SQLite and a server that is tailored to it, the Product +enables the use of CouchDB with SQL queries. +The Licence shall not permit sub-licensing, resale or transfer of the Product to third +parties, other than sub-licensing to the Customer’s direct contractors for the purposes +of utilizing the Product as contemplated above. +The Licence shall not permit the adaptation, modification, decompilation, reverse +engineering or similar activities with respect to the Product. +This licence is granted to the Customer only, although Customer and its Affiliates’ +employees, servants and agents shall be entitled to utilize the Product within the scope +of the Licence for the Customer’s Purpose only. +Reproduction is not permitted to users, except for reproductions that are necessary for +the use of the product under the licence described above. These conditions apply to the +product regardless of the form in which we make the product available and on which +devices it is installed and/or with which devices it is ultimately used. Depending on the +product variant or intended use, certain technical requirements in the IT infrastructure +must be satisfied as a prerequisite for use. +The law of the Northern Ireland applies exclusively to this licence, and the courts of +Northern Ireland shall have exclusive jurisdiction, save that we reserve a right to sue +you in the jurisdiction in which you are based. The application of the UN Sales +Convention (CISG) is excluded. +The invalidity of any part of this licence does not affect the validity of the remaining +regulations. diff --git a/charts/budibase/Chart.lock b/charts/budibase/Chart.lock index 75b9de07b5..3ee752a362 100644 --- a/charts/budibase/Chart.lock +++ b/charts/budibase/Chart.lock @@ -1,9 +1,6 @@ dependencies: - name: couchdb repository: https://apache.github.io/couchdb-helm - version: 3.3.4 -- name: ingress-nginx - repository: https://kubernetes.github.io/ingress-nginx - version: 4.0.13 -digest: sha256:20892705c2d8e64c98257d181063a514ac55013e2b43399a6e54868a97f97845 -generated: "2021-12-30T18:55:30.878411Z" + version: 4.3.0 +digest: sha256:94449a7f195b186f5af33ec5aa66d58b36bede240fae710f021ca87837b30606 +generated: "2023-11-20T17:43:02.777596Z" diff --git a/charts/budibase/Chart.yaml b/charts/budibase/Chart.yaml index 05b3f24dbd..e2c9378f2c 100644 --- a/charts/budibase/Chart.yaml +++ b/charts/budibase/Chart.yaml @@ -17,10 +17,6 @@ version: 0.0.0 appVersion: 0.0.0 dependencies: - name: couchdb - version: 3.3.4 + version: 4.3.0 repository: https://apache.github.io/couchdb-helm condition: services.couchdb.enabled - - name: ingress-nginx - version: 4.0.13 - repository: https://kubernetes.github.io/ingress-nginx - condition: ingress.nginx diff --git a/charts/budibase/README.md b/charts/budibase/README.md index efa78ba75c..d8191026ce 100644 --- a/charts/budibase/README.md +++ b/charts/budibase/README.md @@ -1,39 +1,217 @@ -# Budibase +# budibase -[Budibase](https://budibase.com/) Budibase is an open source low-code platform, helping thousands of teams build apps for their workplace in minutes. - -## TL;DR; -```console -$ cd chart -$ helm install budibase . -``` - -## Introduction - -This chart bootstraps a [Budibase](https://budibase.com/) deployment on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager. +Budibase is an open source low-code platform, helping thousands of teams build apps for their workplace in minutes. ## Prerequisites -- helm v3 or above +- `helm` v3 or above - Kubernetes 1.4+ -- PV provisioner support in the underlying infrastructure (with persistence storage enabled) +- A storage controller (if you want to use persistent storage) +- An ingress controller (if you want to define an `Ingress` resource) +- `metrics-server` (if you want to make use of horizontal pod autoscaling) -## Installing the Chart +## Chart dependencies -To install the chart with the release name `budi-release`: +This chart depends on the official Apache CouchDB chart. You can see its +documentation here: +. + +## Upgrading + +### `2.x` to `3.0.0` + +We made a number of breaking changes in this release to make the chart more +idiomatic and easier to use. + +1. We no longer bundle `ingress-nginx`. If you were relying on this to supply + an ingress controller to your cluster, you will now need to deploy that + separately. You'll find guidance for that here: + . +2. We've upgraded the version of the [CouchDB chart](https://github.com/apache/couchdb-helm) + we use from `3.3.4` to `4.3.0`. The primary motivation for this was to align + the CouchDB chart used with the CouchDB version used, which has also updated + from 3.1.1 to 3.2.1. Additionally, we're moving away from the official CouchDB + to one we're building ourselves. +3. We've separated out the supplied AWS ALB ingress resource for those deploying + into EKS. Where previously you enabled this by setting `ingress.enabled: false` + and `ingress.aws: true`, you now set `awsAlbIngress.enabled: true` and all + configuration for it is under `awsAlbIngress`. +4. The `HorizontalPodAutoscaler` that was configured at `hpa.enabled: true` has + been split into 3 separate HPAs, one for each of `apps`, `worker`, and `proxy`. + They are configured at `services.{apps,worker,proxy}.autoscaling`. + +## Installing + +To install the chart from our repository: ```console -$ helm install budi-release . +$ helm repo add budibase https://budibase.github.io/budibase/ +$ helm repo update +$ helm install --create-namespace --namespace budibase budibase budibase/budibase ``` -The command deploys Budibase on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. - -> **Tip**: List all releases using `helm list` - -## Uninstalling the Chart - -To uninstall/delete the `my-release` deployment: +To install the chart from this repo: ```console -$ helm delete my-release +$ git clone git@github.com:budibase/budibase.git +$ cd budibase/charts/budibase +$ helm install --create-namespace --namespace budibase budibase . ``` + +## Example minimal configuration + +Here's an example `values.yaml` that would get a Budibase instance running in a home +cluster using an nginx ingress controller and NFS as cluster storage (basically one of our +staff's homelabs). + +
+ +```yaml +ingress: + enabled: true + className: "nginx" + hosts: + - host: budibase.local # set this to whatever DNS name you'd use + paths: + - backend: + service: + name: proxy-service + port: + number: 10000 + path: / + pathType: Prefix + +couchdb: + persistentVolume: + enabled: true + storageClass: "nfs-client" + adminPassword: admin + +services: + objectStore: + storageClass: "nfs-client" + redis: + storageClass: "nfs-client" +``` + +If you wanted to use this when bringing up Budibase in your own cluster, you could save it +to your hard disk and run the following: + +```console +$ helm install --create-namespace --namespace budibase budibase . -f values.yaml +``` + +
+ +## Configuring + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| affinity | object | `{}` | Sets the affinity for all pods created by this chart. Should not ordinarily need to be changed. See for more information on affinity. | +| awsAlbIngress.certificateArn | string | `""` | If you're wanting to use HTTPS, you'll need to create an ACM certificate and specify the ARN here. | +| awsAlbIngress.enabled | bool | `false` | Whether to create an ALB Ingress resource pointing to the Budibase proxy. Requires the AWS ALB Ingress Controller. | +| couchdb.clusterSize | int | `1` | The number of replicas to run in the CouchDB cluster. We set this to 1 by default to make things simpler, but you can set it to 3 if you need a high-availability CouchDB cluster. | +| couchdb.couchdbConfig.couchdb.uuid | string | `"budibase-couchdb"` | Unique identifier for this CouchDB server instance. You shouldn't need to change this. | +| couchdb.image | object | `{}` | We use a custom CouchDB image for running Budibase and we don't support using any other CouchDB image. You shouldn't change this, and if you do we can't guarantee that Budibase will work. | +| globals.apiEncryptionKey | string | `""` | Used for encrypting API keys and environment variables when stored in the database. You don't need to set this if `createSecrets` is true. | +| globals.appVersion | string | `""` | The version of Budibase to deploy. Defaults to what's specified by {{ .Chart.AppVersion }}. Ends up being used as the image version tag for the apps, proxy, and worker images. | +| globals.automationMaxIterations | string | `"200"` | The maximum number of iterations allows for an automation loop step. You can read more about looping here: . | +| globals.budibaseEnv | string | `"PRODUCTION"` | Sets the environment variable BUDIBASE_ENVIRONMENT for the apps and worker pods. Should not ordinarily need to be changed. | +| globals.cookieDomain | string | `""` | Sets the domain attribute of the cookie that Budibase uses to store session information. See for details on why you might want to set this. | +| globals.createSecrets | bool | `true` | Create an internal API key, JWT secret, object store access key and secret, and store them in a Kubernetes `Secret`. | +| globals.enableAnalytics | string | `"1"` | Whether to enable analytics or not. You can read more about our analytics here: . | +| globals.google | object | `{"clientId":"","secret":""}` | Google OAuth settings. These can also be set in the Budibase UI, see for details. | +| globals.google.clientId | string | `""` | Client ID of your Google OAuth app. | +| globals.google.secret | string | `""` | Client secret of your Google OAuth app. | +| globals.httpMigrations | string | `"0"` | Whether or not to enable doing data migrations over the HTTP API. If this is set to "0", migrations are run on startup. You shouldn't ordinarily need to change this. | +| globals.internalApiKey | string | `""` | API key used for internal Budibase API calls. You don't need to set this if `createSecrets` is true. | +| globals.internalApiKeyFallback | string | `""` | A fallback value for `internalApiKey`. If you're rotating your encryption key, you can set this to the old value for the duration of the rotation. | +| globals.jwtSecret | string | `""` | Secret used for signing JWTs. You don't need to set this if `createSecrets` is true. | +| globals.jwtSecretFallback | string | `""` | A fallback value for `jwtSecret`. If you're rotating your JWT secret, you can set this to the old value for the duration of the rotation. | +| globals.platformUrl | string | `""` | Set the `platformUrl` binding. You can also do this in Settings > Organisation if you are self-hosting. | +| globals.smtp.enabled | bool | `false` | Whether to enable SMTP or not. | +| globals.smtp.from | string | `""` | The email address to use in the "From:" field of emails sent by Budibase. | +| globals.smtp.host | string | `""` | The hostname of your SMTP server. | +| globals.smtp.password | string | `""` | The password to use when authenticating with your SMTP server. | +| globals.smtp.port | string | `"587"` | The port of your SMTP server. | +| globals.smtp.user | string | `""` | The username to use when authenticating with your SMTP server. | +| globals.tenantFeatureFlags | string | `"*:LICENSING,*:USER_GROUPS,*:ONBOARDING_TOUR"` | Sets what feature flags are enabled and for which tenants. Should not ordinarily need to be changed. | +| imagePullSecrets | list | `[]` | Passed to all pods created by this chart. Should not ordinarily need to be changed. | +| ingress.className | string | `""` | What ingress class to use. | +| ingress.enabled | bool | `true` | Whether to create an Ingress resource pointing to the Budibase proxy. | +| ingress.hosts | list | `[]` | Standard hosts block for the Ingress resource. Defaults to pointing to the Budibase proxy. | +| nameOverride | string | `""` | Override the name of the deploymen. Defaults to {{ .Chart.Name }}. | +| service.port | int | `10000` | Port to expose on the service. | +| service.type | string | `"ClusterIP"` | Service type for the service that points to the main Budibase proxy pod. | +| serviceAccount.annotations | object | `{}` | Annotations to add to the service account | +| serviceAccount.create | bool | `true` | Specifies whether a service account should be created | +| serviceAccount.name | string | `""` | The name of the service account to use. If not set and create is true, a name is generated using the fullname template | +| services.apps.autoscaling.enabled | bool | `false` | Whether to enable horizontal pod autoscaling for the apps service. | +| services.apps.autoscaling.maxReplicas | int | `10` | | +| services.apps.autoscaling.minReplicas | int | `1` | | +| services.apps.autoscaling.targetCPUUtilizationPercentage | int | `80` | Target CPU utilization percentage for the apps service. Note that for autoscaling to work, you will need to have metrics-server configured, and resources set for the apps pods. | +| services.apps.httpLogging | int | `1` | Whether or not to log HTTP requests to the apps service. | +| services.apps.livenessProbe | object | HTTP health checks. | Liveness probe configuration for apps pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.apps.logLevel | string | `"info"` | The log level for the apps service. | +| services.apps.readinessProbe | object | HTTP health checks. | Readiness probe configuration for apps pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.apps.replicaCount | int | `1` | The number of apps replicas to run. | +| services.apps.resources | object | `{}` | The resources to use for apps pods. See for more information on how to set these. | +| services.apps.startupProbe | object | HTTP health checks. | Startup probe configuration for apps pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.couchdb.backup.enabled | bool | `false` | Whether or not to enable periodic CouchDB backups. This works by replicating to another CouchDB instance. | +| services.couchdb.backup.interval | string | `""` | Backup interval in seconds | +| services.couchdb.backup.resources | object | `{}` | The resources to use for CouchDB backup pods. See for more information on how to set these. | +| services.couchdb.backup.target | string | `""` | Target couchDB instance to back up to, either a hostname or an IP address. | +| services.couchdb.enabled | bool | `true` | Whether or not to spin up a CouchDB instance in your cluster. True by default, and the configuration for the CouchDB instance is under the `couchdb` key at the root of this file. You can see what options are available to you by looking at the official CouchDB Helm chart: . | +| services.couchdb.port | int | `5984` | | +| services.dns | string | `"cluster.local"` | The DNS suffix to use for service discovery. You only need to change this if you've configured your cluster to use a different DNS suffix. | +| services.objectStore.accessKey | string | `""` | AWS_ACCESS_KEY if using S3 | +| services.objectStore.browser | bool | `true` | Whether to enable the Minio web console or not. If you're exposing Minio to the Internet (via a custom Ingress record, for example), you should set this to false. If you're only exposing Minio to your cluster, you can leave this as true. | +| services.objectStore.cloudfront.cdn | string | `""` | Set the url of a distribution to enable cloudfront. | +| services.objectStore.cloudfront.privateKey64 | string | `""` | Base64 encoded private key for the above public key. | +| services.objectStore.cloudfront.publicKeyId | string | `""` | ID of public key stored in cloudfront. | +| services.objectStore.minio | bool | `true` | Set to false if using another object store, such as S3. You will need to set `services.objectStore.url` to point to your bucket if you do this. | +| services.objectStore.region | string | `""` | AWS_REGION if using S3 | +| services.objectStore.resources | object | `{}` | The resources to use for Minio pods. See for more information on how to set these. | +| services.objectStore.secretKey | string | `""` | AWS_SECRET_ACCESS_KEY if using S3 | +| services.objectStore.storage | string | `"100Mi"` | How much storage to give Minio in its PersistentVolumeClaim. | +| services.objectStore.storageClass | string | `""` | If defined, storageClassName: If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. | +| services.objectStore.url | string | `"http://minio-service:9000"` | URL to use for object storage. Only change this if you're using an external object store, such as S3. Remember to set `minio: false` if you do this. | +| services.proxy.autoscaling.enabled | bool | `false` | Whether to enable horizontal pod autoscaling for the proxy service. | +| services.proxy.autoscaling.maxReplicas | int | `10` | | +| services.proxy.autoscaling.minReplicas | int | `1` | | +| services.proxy.autoscaling.targetCPUUtilizationPercentage | int | `80` | Target CPU utilization percentage for the proxy service. Note that for autoscaling to work, you will need to have metrics-server configured, and resources set for the proxy pods. | +| services.proxy.livenessProbe | object | HTTP health checks. | Liveness probe configuration for proxy pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.proxy.readinessProbe | object | HTTP health checks. | Readiness probe configuration for proxy pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.proxy.replicaCount | int | `1` | The number of proxy replicas to run. | +| services.proxy.resources | object | `{}` | The resources to use for proxy pods. See for more information on how to set these. | +| services.proxy.startupProbe | object | HTTP health checks. | Startup probe configuration for proxy pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.redis.enabled | bool | `true` | Whether or not to deploy a Redis pod into your cluster. | +| services.redis.password | string | `"budibase"` | The password to use when connecting to Redis. It's recommended that you change this from the default if you're running Redis in-cluster. | +| services.redis.port | int | `6379` | Port to expose Redis on. | +| services.redis.resources | object | `{}` | The resources to use for Redis pods. See for more information on how to set these. | +| services.redis.storage | string | `"100Mi"` | How much persistent storage to allocate to Redis. | +| services.redis.storageClass | string | `""` | If defined, storageClassName: If set to "-", storageClassName: "", which disables dynamic provisioning If undefined (the default) or set to null, no storageClassName spec is set, choosing the default provisioner. | +| services.redis.url | string | `""` | If you choose to run Redis externally to this chart, you can specify the connection details here. | +| services.worker.autoscaling.enabled | bool | `false` | Whether to enable horizontal pod autoscaling for the worker service. | +| services.worker.autoscaling.maxReplicas | int | `10` | | +| services.worker.autoscaling.minReplicas | int | `1` | | +| services.worker.autoscaling.targetCPUUtilizationPercentage | int | `80` | Target CPU utilization percentage for the worker service. Note that for autoscaling to work, you will need to have metrics-server configured, and resources set for the worker pods. | +| services.worker.httpLogging | int | `1` | Whether or not to log HTTP requests to the worker service. | +| services.worker.livenessProbe | object | HTTP health checks. | Liveness probe configuration for worker pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.worker.logLevel | string | `"info"` | The log level for the worker service. | +| services.worker.readinessProbe | object | HTTP health checks. | Readiness probe configuration for worker pods. You shouldn't need to change this, but if you want to you can find more information here: | +| services.worker.replicaCount | int | `1` | The number of worker replicas to run. | +| services.worker.resources | object | `{}` | The resources to use for worker pods. See for more information on how to set these. | +| services.worker.startupProbe | object | HTTP health checks. | Startup probe configuration for worker pods. You shouldn't need to change this, but if you want to you can find more information here: | +| tolerations | list | `[]` | Sets the tolerations for all pods created by this chart. Should not ordinarily need to be changed. See for more information on tolerations. | + +## Uninstalling + +To uninstall the chart, assuming you named the release `budibase` (both commands in the installation section do so): + +```console +$ helm uninstall --namespace budibase budibase +``` + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.11.3](https://github.com/norwoodj/helm-docs/releases/v1.11.3) diff --git a/charts/budibase/README.md.gotmpl b/charts/budibase/README.md.gotmpl new file mode 100644 index 0000000000..e37c323837 --- /dev/null +++ b/charts/budibase/README.md.gotmpl @@ -0,0 +1,117 @@ +{{ template "chart.header" . }} +{{ template "chart.description" . }} + +## Prerequisites + +- `helm` v3 or above +- Kubernetes 1.4+ +- A storage controller (if you want to use persistent storage) +- An ingress controller (if you want to define an `Ingress` resource) +- `metrics-server` (if you want to make use of horizontal pod autoscaling) + +## Chart dependencies + +This chart depends on the official Apache CouchDB chart. You can see its +documentation here: +. + +## Upgrading + +### `2.x` to `3.0.0` + +We made a number of breaking changes in this release to make the chart more +idiomatic and easier to use. + +1. We no longer bundle `ingress-nginx`. If you were relying on this to supply + an ingress controller to your cluster, you will now need to deploy that + separately. You'll find guidance for that here: + . +2. We've upgraded the version of the [CouchDB chart](https://github.com/apache/couchdb-helm) + we use from `3.3.4` to `4.3.0`. The primary motivation for this was to align + the CouchDB chart used with the CouchDB version used, which has also updated + from 3.1.1 to 3.2.1. Additionally, we're moving away from the official CouchDB + to one we're building ourselves. +3. We've separated out the supplied AWS ALB ingress resource for those deploying + into EKS. Where previously you enabled this by setting `ingress.enabled: false` + and `ingress.aws: true`, you now set `awsAlbIngress.enabled: true` and all + configuration for it is under `awsAlbIngress`. +4. The `HorizontalPodAutoscaler` that was configured at `hpa.enabled: true` has + been split into 3 separate HPAs, one for each of `apps`, `worker`, and `proxy`. + They are configured at `services.{apps,worker,proxy}.autoscaling`. + +## Installing + +To install the chart from our repository: + +```console +$ helm repo add budibase https://budibase.github.io/budibase/ +$ helm repo update +$ helm install --create-namespace --namespace budibase budibase budibase/budibase +``` + +To install the chart from this repo: + +```console +$ git clone git@github.com:budibase/budibase.git +$ cd budibase/charts/budibase +$ helm install --create-namespace --namespace budibase budibase . +``` + +## Example minimal configuration + +Here's an example `values.yaml` that would get a Budibase instance running in a home +cluster using an nginx ingress controller and NFS as cluster storage (basically one of our +staff's homelabs). + +
+ +```yaml +ingress: + enabled: true + className: "nginx" + hosts: + - host: budibase.local # set this to whatever DNS name you'd use + paths: + - backend: + service: + name: proxy-service + port: + number: 10000 + path: / + pathType: Prefix + +couchdb: + persistentVolume: + enabled: true + storageClass: "nfs-client" + adminPassword: admin + +services: + objectStore: + storageClass: "nfs-client" + redis: + storageClass: "nfs-client" +``` + +If you wanted to use this when bringing up Budibase in your own cluster, you could save it +to your hard disk and run the following: + +```console +$ helm install --create-namespace --namespace budibase budibase . -f values.yaml +``` + +
+ +## Configuring + +{{ template "chart.valuesTable" . }} + +## Uninstalling + +To uninstall the chart, assuming you named the release `budibase` (both commands in the installation section do so): + +```console +$ helm uninstall --namespace budibase budibase +``` + +{{ template "helm-docs.versionFooter" . }} diff --git a/charts/budibase/charts/couchdb-3.3.4.tgz b/charts/budibase/charts/couchdb-3.3.4.tgz deleted file mode 100644 index f7ebfd3e96..0000000000 Binary files a/charts/budibase/charts/couchdb-3.3.4.tgz and /dev/null differ diff --git a/charts/budibase/charts/couchdb-4.3.0.tgz b/charts/budibase/charts/couchdb-4.3.0.tgz new file mode 100644 index 0000000000..d3cce28ee6 Binary files /dev/null and b/charts/budibase/charts/couchdb-4.3.0.tgz differ diff --git a/charts/budibase/charts/ingress-nginx-4.0.13.tgz b/charts/budibase/charts/ingress-nginx-4.0.13.tgz deleted file mode 100644 index 1e34215c5f..0000000000 Binary files a/charts/budibase/charts/ingress-nginx-4.0.13.tgz and /dev/null differ diff --git a/charts/budibase/templates/alb-ingress.yaml b/charts/budibase/templates/alb-ingress.yaml index fd38364ba6..89b4e9e2cb 100644 --- a/charts/budibase/templates/alb-ingress.yaml +++ b/charts/budibase/templates/alb-ingress.yaml @@ -1,4 +1,4 @@ -{{- if .Values.ingress.aws }} +{{- if .Values.awsAlbIngress.enabled }} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: @@ -9,22 +9,22 @@ metadata: alb.ingress.kubernetes.io/target-type: ip alb.ingress.kubernetes.io/success-codes: '200' alb.ingress.kubernetes.io/healthcheck-path: '/health' - {{- if .Values.ingress.certificateArn }} + {{- if .Values.awsAlbIngress.certificateArn }} alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' - alb.ingress.kubernetes.io/certificate-arn: {{ .Values.ingress.certificateArn }} + alb.ingress.kubernetes.io/certificate-arn: {{ .Values.awsAlbIngress.certificateArn }} {{- end }} - {{- if .Values.ingress.sslPolicy }} - alb.ingress.kubernetes.io/actions.ssl-policy: {{ .Values.ingress.sslPolicy }} + {{- if .Values.awsAlbIngress.sslPolicy }} + alb.ingress.kubernetes.io/actions.ssl-policy: {{ .Values.awsAlbIngress.sslPolicy }} {{- end }} - {{- if .Values.ingress.securityGroups }} - alb.ingress.kubernetes.io/security-groups: {{ .Values.ingress.securityGroups }} + {{- if .Values.awsAlbIngress.securityGroups }} + alb.ingress.kubernetes.io/security-groups: {{ .Values.awsAlbIngress.securityGroups }} {{- end }} spec: rules: - http: paths: - {{- if .Values.ingress.certificateArn }} + {{- if .Values.awsAlbIngress.certificateArn }} - path: / pathType: Prefix backend: diff --git a/charts/budibase/templates/app-service-deployment.yaml b/charts/budibase/templates/app-service-deployment.yaml index 73c6d990d2..7358e474ca 100644 --- a/charts/budibase/templates/app-service-deployment.yaml +++ b/charts/budibase/templates/app-service-deployment.yaml @@ -2,12 +2,9 @@ apiVersion: apps/v1 kind: Deployment metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.apps.deploymentAnnotations }} {{- toYaml .Values.services.apps.deploymentAnnotations | indent 4 -}} {{ end }} - creationTimestamp: null labels: io.kompose.service: app-service {{ if .Values.services.apps.deploymentLabels }} @@ -24,12 +21,9 @@ spec: template: metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.apps.templateAnnotations }} {{- toYaml .Values.services.apps.templateAnnotations | indent 8 -}} {{ end }} - creationTimestamp: null labels: io.kompose.service: app-service {{ if .Values.services.apps.templateLabels }} diff --git a/charts/budibase/templates/app-service-hpa.yaml b/charts/budibase/templates/app-service-hpa.yaml new file mode 100644 index 0000000000..e819ecb9e3 --- /dev/null +++ b/charts/budibase/templates/app-service-hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.services.apps.autoscaling.enabled }} +apiVersion: {{ ternary "autoscaling/v2" "autoscaling/v2beta2" (.Capabilities.APIVersions.Has "autoscaling/v2") }} +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "budibase.fullname" . }}-apps + labels: + {{- include "budibase.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: app-service + minReplicas: {{ .Values.services.apps.autoscaling.minReplicas }} + maxReplicas: {{ .Values.services.apps.autoscaling.maxReplicas }} + metrics: + {{- if .Values.services.apps.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.services.apps.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.services.apps.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.services.apps.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/charts/budibase/templates/app-service-service.yaml b/charts/budibase/templates/app-service-service.yaml index 5247b4de09..6d19590d45 100644 --- a/charts/budibase/templates/app-service-service.yaml +++ b/charts/budibase/templates/app-service-service.yaml @@ -1,10 +1,6 @@ apiVersion: v1 kind: Service metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: app-service name: app-service diff --git a/charts/budibase/templates/couchdb-backup.yaml b/charts/budibase/templates/couchdb-backup.yaml index 7396f97476..6f842537a7 100644 --- a/charts/budibase/templates/couchdb-backup.yaml +++ b/charts/budibase/templates/couchdb-backup.yaml @@ -2,10 +2,6 @@ apiVersion: apps/v1 kind: Deployment metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: app.kubernetes.io/name: couchdb-backup name: couchdb-backup @@ -18,10 +14,6 @@ spec: type: Recreate template: metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: app.kubernetes.io/name: couchdb-backup spec: diff --git a/charts/budibase/templates/hpa.yaml b/charts/budibase/templates/hpa.yaml deleted file mode 100644 index 2f901b4664..0000000000 --- a/charts/budibase/templates/hpa.yaml +++ /dev/null @@ -1,28 +0,0 @@ -{{- if .Values.autoscaling.enabled }} -apiVersion: autoscaling/v2beta1 -kind: HorizontalPodAutoscaler -metadata: - name: {{ include "budibase.fullname" . }} - labels: - {{- include "budibase.labels" . | nindent 4 }} -spec: - scaleTargetRef: - apiVersion: apps/v1 - kind: Deployment - name: {{ include "budibase.fullname" . }} - minReplicas: {{ .Values.autoscaling.minReplicas }} - maxReplicas: {{ .Values.autoscaling.maxReplicas }} - metrics: - {{- if .Values.autoscaling.targetCPUUtilizationPercentage }} - - type: Resource - resource: - name: cpu - targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }} - {{- end }} - {{- if .Values.autoscaling.targetMemoryUtilizationPercentage }} - - type: Resource - resource: - name: memory - targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }} - {{- end }} -{{- end }} diff --git a/charts/budibase/templates/minio-data-persistentvolumeclaim.yaml b/charts/budibase/templates/minio-data-persistentvolumeclaim.yaml index abcf341bc5..c17001a436 100644 --- a/charts/budibase/templates/minio-data-persistentvolumeclaim.yaml +++ b/charts/budibase/templates/minio-data-persistentvolumeclaim.yaml @@ -2,7 +2,6 @@ apiVersion: v1 kind: PersistentVolumeClaim metadata: - creationTimestamp: null labels: io.kompose.service: minio-data name: minio-data diff --git a/charts/budibase/templates/minio-service-deployment.yaml b/charts/budibase/templates/minio-service-deployment.yaml index f98ecc139d..28e8eb9991 100644 --- a/charts/budibase/templates/minio-service-deployment.yaml +++ b/charts/budibase/templates/minio-service-deployment.yaml @@ -2,10 +2,6 @@ apiVersion: apps/v1 kind: Deployment metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: minio-service name: minio-service @@ -18,10 +14,6 @@ spec: type: Recreate template: metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: minio-service spec: diff --git a/charts/budibase/templates/minio-service-service.yaml b/charts/budibase/templates/minio-service-service.yaml index cfdb22002b..ce89f1347c 100644 --- a/charts/budibase/templates/minio-service-service.yaml +++ b/charts/budibase/templates/minio-service-service.yaml @@ -2,10 +2,6 @@ apiVersion: v1 kind: Service metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: minio-service name: minio-service diff --git a/charts/budibase/templates/proxy-service-deployment.yaml b/charts/budibase/templates/proxy-service-deployment.yaml index 53bba6232d..706e9b4b73 100644 --- a/charts/budibase/templates/proxy-service-deployment.yaml +++ b/charts/budibase/templates/proxy-service-deployment.yaml @@ -2,12 +2,9 @@ apiVersion: apps/v1 kind: Deployment metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.proxy.deploymentAnnotations }} {{- toYaml .Values.services.proxy.deploymentAnnotations | indent 4 -}} {{ end }} - creationTimestamp: null labels: app.kubernetes.io/name: budibase-proxy {{ if .Values.services.proxy.deploymentLabels }} @@ -19,17 +16,15 @@ spec: selector: matchLabels: app.kubernetes.io/name: budibase-proxy + minReadySeconds: 10 strategy: type: RollingUpdate template: metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.proxy.templateAnnotations }} {{- toYaml .Values.services.proxy.templateAnnotations | indent 8 -}} {{ end }} - creationTimestamp: null labels: app.kubernetes.io/name: budibase-proxy {{ if .Values.services.proxy.templateLabels }} diff --git a/charts/budibase/templates/proxy-service-hpa.yaml b/charts/budibase/templates/proxy-service-hpa.yaml new file mode 100644 index 0000000000..b6c6022008 --- /dev/null +++ b/charts/budibase/templates/proxy-service-hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.services.proxy.autoscaling.enabled }} +apiVersion: {{ ternary "autoscaling/v2" "autoscaling/v2beta2" (.Capabilities.APIVersions.Has "autoscaling/v2") }} +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "budibase.fullname" . }}-proxy + labels: + {{- include "budibase.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: proxy-service + minReplicas: {{ .Values.services.proxy.autoscaling.minReplicas }} + maxReplicas: {{ .Values.services.proxy.autoscaling.maxReplicas }} + metrics: + {{- if .Values.services.proxy.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.services.proxy.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.services.proxy.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.services.proxy.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/charts/budibase/templates/proxy-service-service.yaml b/charts/budibase/templates/proxy-service-service.yaml index bf2b199ee5..988c540599 100644 --- a/charts/budibase/templates/proxy-service-service.yaml +++ b/charts/budibase/templates/proxy-service-service.yaml @@ -1,10 +1,6 @@ apiVersion: v1 kind: Service metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: app.kubernetes.io/name: budibase-proxy name: proxy-service @@ -16,4 +12,4 @@ spec: selector: app.kubernetes.io/name: budibase-proxy status: - loadBalancer: {} \ No newline at end of file + loadBalancer: {} diff --git a/charts/budibase/templates/redis-data-persistentvolumeclaim.yaml b/charts/budibase/templates/redis-data-persistentvolumeclaim.yaml index 6f11492b58..e605a98376 100644 --- a/charts/budibase/templates/redis-data-persistentvolumeclaim.yaml +++ b/charts/budibase/templates/redis-data-persistentvolumeclaim.yaml @@ -2,7 +2,6 @@ apiVersion: v1 kind: PersistentVolumeClaim metadata: - creationTimestamp: null labels: io.kompose.service: redis-data name: redis-data diff --git a/charts/budibase/templates/redis-service-deployment.yaml b/charts/budibase/templates/redis-service-deployment.yaml index 9b39d14291..bca40d2237 100644 --- a/charts/budibase/templates/redis-service-deployment.yaml +++ b/charts/budibase/templates/redis-service-deployment.yaml @@ -2,10 +2,6 @@ apiVersion: apps/v1 kind: Deployment metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: redis-service name: redis-service @@ -18,10 +14,6 @@ spec: type: Recreate template: metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: redis-service spec: diff --git a/charts/budibase/templates/redis-service-service.yaml b/charts/budibase/templates/redis-service-service.yaml index 55ca40ed88..b8f998925a 100644 --- a/charts/budibase/templates/redis-service-service.yaml +++ b/charts/budibase/templates/redis-service-service.yaml @@ -2,10 +2,6 @@ apiVersion: v1 kind: Service metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: redis-service name: redis-service diff --git a/charts/budibase/templates/worker-service-deployment.yaml b/charts/budibase/templates/worker-service-deployment.yaml index 5e0addb9dd..6427aa70e8 100644 --- a/charts/budibase/templates/worker-service-deployment.yaml +++ b/charts/budibase/templates/worker-service-deployment.yaml @@ -2,12 +2,9 @@ apiVersion: apps/v1 kind: Deployment metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.worker.deploymentAnnotations }} {{- toYaml .Values.services.worker.deploymentAnnotations | indent 4 -}} {{ end }} - creationTimestamp: null labels: io.kompose.service: worker-service {{ if .Values.services.worker.deploymentLabels }} @@ -24,12 +21,9 @@ spec: template: metadata: annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) {{ if .Values.services.worker.templateAnnotations }} {{- toYaml .Values.services.worker.templateAnnotations | indent 8 -}} {{ end }} - creationTimestamp: null labels: io.kompose.service: worker-service {{ if .Values.services.worker.templateLabels }} diff --git a/charts/budibase/templates/worker-service-hpa.yaml b/charts/budibase/templates/worker-service-hpa.yaml new file mode 100644 index 0000000000..a04cc259a0 --- /dev/null +++ b/charts/budibase/templates/worker-service-hpa.yaml @@ -0,0 +1,32 @@ +{{- if .Values.services.worker.autoscaling.enabled }} +apiVersion: {{ ternary "autoscaling/v2" "autoscaling/v2beta2" (.Capabilities.APIVersions.Has "autoscaling/v2") }} +kind: HorizontalPodAutoscaler +metadata: + name: {{ include "budibase.fullname" . }}-worker + labels: + {{- include "budibase.labels" . | nindent 4 }} +spec: + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: worker-service + minReplicas: {{ .Values.services.worker.autoscaling.minReplicas }} + maxReplicas: {{ .Values.services.worker.autoscaling.maxReplicas }} + metrics: + {{- if .Values.services.worker.autoscaling.targetCPUUtilizationPercentage }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.services.worker.autoscaling.targetCPUUtilizationPercentage }} + {{- end }} + {{- if .Values.services.worker.autoscaling.targetMemoryUtilizationPercentage }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.services.worker.autoscaling.targetMemoryUtilizationPercentage }} + {{- end }} +{{- end }} diff --git a/charts/budibase/templates/worker-service-service.yaml b/charts/budibase/templates/worker-service-service.yaml index a79ba1e04b..c5f56ba205 100644 --- a/charts/budibase/templates/worker-service-service.yaml +++ b/charts/budibase/templates/worker-service-service.yaml @@ -1,10 +1,6 @@ apiVersion: v1 kind: Service metadata: - annotations: - kompose.cmd: kompose convert - kompose.version: 1.21.0 (992df58d8) - creationTimestamp: null labels: io.kompose.service: worker-service name: worker-service diff --git a/charts/budibase/values.yaml b/charts/budibase/values.yaml index 857067d0f1..13054e75fc 100644 --- a/charts/budibase/values.yaml +++ b/charts/budibase/values.yaml @@ -1,56 +1,32 @@ -# Default values for budibase. -# This is a YAML-formatted file. -# Declare variables to be passed into your templates. - -image: - pullPolicy: IfNotPresent - # Overrides the image tag whose default is the chart appVersion. - tag: "" - +# -- Passed to all pods created by this chart. Should not ordinarily need to be changed. imagePullSecrets: [] +# -- Override the name of the deploymen. Defaults to {{ .Chart.Name }}. nameOverride: "" -# fullnameOverride: "" serviceAccount: - # Specifies whether a service account should be created + # -- Specifies whether a service account should be created create: true - # Annotations to add to the service account + # -- Annotations to add to the service account annotations: {} - # The name of the service account to use. + # -- The name of the service account to use. # If not set and create is true, a name is generated using the fullname template name: "" -podAnnotations: {} - -podSecurityContext: - {} - # fsGroup: 2000 - -securityContext: - {} - # capabilities: - # drop: - # - ALL - # readOnlyRootFilesystem: true - # runAsNonRoot: true - # runAsUser: 1000 - service: + # -- Service type for the service that points to the main Budibase proxy pod. type: ClusterIP + # -- Port to expose on the service. port: 10000 ingress: + # -- Whether to create an Ingress resource pointing to the Budibase proxy. enabled: true - aws: false - nginx: true - certificateArn: "" + # -- What ingress class to use. className: "" - annotations: - kubernetes.io/ingress.class: nginx - nginx.ingress.kubernetes.io/client-max-body-size: 150M - nginx.ingress.kubernetes.io/proxy-body-size: 50m + # -- Standard hosts block for the Ingress resource. Defaults to pointing to the Budibase proxy. hosts: - - host: # change if using custom domain + # @ignore + - host: paths: - path: / pathType: Prefix @@ -60,361 +36,426 @@ ingress: port: number: 10000 -autoscaling: +awsAlbIngress: + # -- Whether to create an ALB Ingress resource pointing to the Budibase proxy. Requires the AWS ALB Ingress Controller. enabled: false - minReplicas: 1 - maxReplicas: 100 - targetCPUUtilizationPercentage: 80 - # targetMemoryUtilizationPercentage: 80 - -nodeSelector: {} + # -- If you're wanting to use HTTPS, you'll need to create an ACM certificate and specify the ARN here. + certificateArn: "" +# -- Sets the tolerations for all pods created by this chart. Should not ordinarily need to be changed. +# See for more information +# on tolerations. tolerations: [] +# -- Sets the affinity for all pods created by this chart. Should not ordinarily +# need to be changed. See +# +# for more information on affinity. affinity: {} globals: - appVersion: "" # Use as an override to .Chart.AppVersion + # -- The version of Budibase to deploy. Defaults to what's specified by {{ .Chart.AppVersion }}. + # Ends up being used as the image version tag for the apps, proxy, and worker images. + appVersion: "" + # -- Sets the environment variable BUDIBASE_ENVIRONMENT for the apps and worker pods. Should not + # ordinarily need to be changed. budibaseEnv: PRODUCTION + # -- Sets what feature flags are enabled and for which tenants. Should not ordinarily need to be + # changed. tenantFeatureFlags: "*:LICENSING,*:USER_GROUPS,*:ONBOARDING_TOUR" + # -- Whether to enable analytics or not. You can read more about our analytics here: + # . enableAnalytics: "1" + # @ignore (only used if enableAnalytics is set to 1) posthogToken: "phc_bIjZL7oh2GEUd2vqvTBH8WvrX0fWTFQMs6H5KQxiUxU" - selfHosted: "1" # set to 0 for budibase cloud environment, set to 1 for self-hosted setup - multiTenancy: "0" # set to 0 to disable multiple orgs, set to 1 to enable multiple orgs - offlineMode: "0" # set to 1 to enable offline mode + # @ignore (should not normally need to be changed, we only set this to "0" + # when deploying to our Cloud environment) + selfHosted: "1" + # @ignore (doesn't work out of the box for self-hosted users, only meant for Budicloud) + multiTenancy: "0" + # @ignore (only currently used to determine whether to fetch licenses offline or not, should + # not normally need to be changed, and only applies to Enterprise customers) + offlineMode: "0" + # @ignore (only needs to be set in our cloud environment) accountPortalUrl: "" + # @ignore (only needs to be set in our cloud environment) accountPortalApiKey: "" + # -- Sets the domain attribute of the cookie that Budibase uses to store session information. + # See + # for details on why you might want to set this. cookieDomain: "" + # -- Set the `platformUrl` binding. You can also do this in Settings > Organisation if you are + # self-hosting. platformUrl: "" + # -- Whether or not to enable doing data migrations over the HTTP API. If this is set to "0", + # migrations are run on startup. You shouldn't ordinarily need to change this. httpMigrations: "0" + # -- Google OAuth settings. These can also be set in the Budibase UI, see + # for details. google: + # -- Client ID of your Google OAuth app. clientId: "" + # -- Client secret of your Google OAuth app. secret: "" + # -- The maximum number of iterations allows for an automation loop step. You can read more about + # looping here: . automationMaxIterations: "200" - createSecrets: true # creates an internal API key, JWT secrets and redis password for you + # -- Create an internal API key, JWT secret, object store access key and + # secret, and store them in a Kubernetes `Secret`. + createSecrets: true - # if createSecrets is set to false, you can hard-code your secrets here + # -- Used for encrypting API keys and environment variables when stored in the database. + # You don't need to set this if `createSecrets` is true. apiEncryptionKey: "" + # -- API key used for internal Budibase API calls. You don't need to set this + # if `createSecrets` is true. internalApiKey: "" + # -- Secret used for signing JWTs. You don't need to set this if `createSecrets` is true. jwtSecret: "" - cdnUrl: "" - # fallback values used during live rotation + + # -- A fallback value for `internalApiKey`. If you're rotating your encryption key, you can + # set this to the old value for the duration of the rotation. internalApiKeyFallback: "" + # -- A fallback value for `jwtSecret`. If you're rotating your JWT secret, you can set this + # to the old value for the duration of the rotation. jwtSecretFallback: "" smtp: + # -- Whether to enable SMTP or not. enabled: false - -# globalAgentHttpProxy: -# globalAgentHttpsProxy: -# globalAgentNoProxy: + # -- The hostname of your SMTP server. + host: "" + # -- The port of your SMTP server. + port: "587" + # -- The email address to use in the "From:" field of emails sent by Budibase. + from: "" + # -- The username to use when authenticating with your SMTP server. + user: "" + # -- The password to use when authenticating with your SMTP server. + password: "" services: - budibaseVersion: latest + # -- The DNS suffix to use for service discovery. You only need to change this + # if you've configured your cluster to use a different DNS suffix. dns: cluster.local - # tlsRejectUnauthorized: 0 proxy: + # @ignore (you shouldn't need to change this) port: 10000 + # -- The number of proxy replicas to run. replicaCount: 1 + # @ignore (you should never need to change this) upstreams: apps: "http://app-service.{{ .Release.Namespace }}.svc.{{ .Values.services.dns }}:{{ .Values.services.apps.port }}" worker: "http://worker-service.{{ .Release.Namespace }}.svc.{{ .Values.services.dns }}:{{ .Values.services.worker.port }}" minio: "http://minio-service.{{ .Release.Namespace }}.svc.{{ .Values.services.dns }}:{{ .Values.services.objectStore.port }}" couchdb: "http://{{ .Release.Name }}-svc-couchdb:{{ .Values.services.couchdb.port }}" + # -- The resources to use for proxy pods. See + # + # for more information on how to set these. resources: {} + # -- Startup probe configuration for proxy pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. startupProbe: + # @ignore httpGet: path: /health port: 10000 scheme: HTTP + # @ignore failureThreshold: 30 + # @ignore periodSeconds: 3 + # -- Readiness probe configuration for proxy pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. readinessProbe: + # @ignore httpGet: path: /health port: 10000 scheme: HTTP + # @ignore periodSeconds: 3 + # @ignore failureThreshold: 1 + # -- Liveness probe configuration for proxy pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. livenessProbe: + # @ignore httpGet: path: /health port: 10000 scheme: HTTP + # @ignore failureThreshold: 3 + # @ignore periodSeconds: 5 - # annotations: - # co.elastic.logs/module: nginx - # co.elastic.logs/fileset.stdout: access - # co.elastic.logs/fileset.stderr: error + autoscaling: + # -- Whether to enable horizontal pod autoscaling for the proxy service. + enabled: false + minReplicas: 1 + maxReplicas: 10 + # -- Target CPU utilization percentage for the proxy service. Note that + # for autoscaling to work, you will need to have metrics-server + # configured, and resources set for the proxy pods. + targetCPUUtilizationPercentage: 80 apps: + # @ignore (you shouldn't need to change this) port: 4002 + # -- The number of apps replicas to run. replicaCount: 1 + # -- The log level for the apps service. logLevel: info + # -- Whether or not to log HTTP requests to the apps service. httpLogging: 1 + # -- The resources to use for apps pods. See + # + # for more information on how to set these. resources: {} + # -- Startup probe configuration for apps pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. startupProbe: + # @ignore httpGet: path: /health port: 4002 scheme: HTTP + # @ignore failureThreshold: 30 + # @ignore periodSeconds: 3 + # -- Readiness probe configuration for apps pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. readinessProbe: + # @ignore httpGet: path: /health port: 4002 scheme: HTTP + # @ignore periodSeconds: 3 + # @ignore failureThreshold: 1 + # -- Liveness probe configuration for apps pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. livenessProbe: + # @ignore httpGet: path: /health port: 4002 scheme: HTTP + # @ignore failureThreshold: 3 + # @ignore periodSeconds: 5 - # nodeDebug: "" # set the value of NODE_DEBUG - # annotations: - # co.elastic.logs/multiline.type: pattern - # co.elastic.logs/multiline.pattern: '^[[:space:]]' - # co.elastic.logs/multiline.negate: false - # co.elastic.logs/multiline.match: after + autoscaling: + # -- Whether to enable horizontal pod autoscaling for the apps service. + enabled: false + minReplicas: 1 + maxReplicas: 10 + # -- Target CPU utilization percentage for the apps service. Note that for + # autoscaling to work, you will need to have metrics-server configured, + # and resources set for the apps pods. + targetCPUUtilizationPercentage: 80 + worker: + # @ignore (you shouldn't need to change this) port: 4003 + # -- The number of worker replicas to run. replicaCount: 1 + # -- The log level for the worker service. logLevel: info + # -- Whether or not to log HTTP requests to the worker service. httpLogging: 1 + # -- The resources to use for worker pods. See + # + # for more information on how to set these. resources: {} + # -- Startup probe configuration for worker pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. startupProbe: + # @ignore httpGet: path: /health port: 4003 scheme: HTTP + # @ignore failureThreshold: 30 + # @ignore periodSeconds: 3 + # -- Readiness probe configuration for worker pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. readinessProbe: + # @ignore httpGet: path: /health port: 4003 scheme: HTTP + # @ignore periodSeconds: 3 + # @ignore failureThreshold: 1 + # -- Liveness probe configuration for worker pods. You shouldn't need to + # change this, but if you want to you can find more information here: + # + # @default -- HTTP health checks. livenessProbe: + # @ignore httpGet: path: /health port: 4003 scheme: HTTP + # @ignore failureThreshold: 3 + # @ignore periodSeconds: 5 - # annotations: - # co.elastic.logs/multiline.type: pattern - # co.elastic.logs/multiline.pattern: '^[[:space:]]' - # co.elastic.logs/multiline.negate: false - # co.elastic.logs/multiline.match: after + autoscaling: + # -- Whether to enable horizontal pod autoscaling for the worker service. + enabled: false + minReplicas: 1 + maxReplicas: 10 + # -- Target CPU utilization percentage for the worker service. Note that + # for autoscaling to work, you will need to have metrics-server + # configured, and resources set for the worker pods. + targetCPUUtilizationPercentage: 80 couchdb: + # -- Whether or not to spin up a CouchDB instance in your cluster. True by + # default, and the configuration for the CouchDB instance is under the + # `couchdb` key at the root of this file. You can see what options are + # available to you by looking at the official CouchDB Helm chart: + # . enabled: true # url: "" # only change if pointing to existing couch server # user: "" # only change if pointing to existing couch server # password: "" # only change if pointing to existing couch server port: 5984 backup: + # -- Whether or not to enable periodic CouchDB backups. This works by replicating + # to another CouchDB instance. enabled: false - # target couchDB instance to back up to + # -- Target couchDB instance to back up to, either a hostname or an IP address. target: "" - # backup interval in seconds + # -- Backup interval in seconds interval: "" + # -- The resources to use for CouchDB backup pods. See + # + # for more information on how to set these. resources: {} redis: - enabled: true # disable if using external redis + # -- Whether or not to deploy a Redis pod into your cluster. + enabled: true + # -- Port to expose Redis on. port: 6379 + # @ignore (you should leave this as 1, we don't support clustering Redis) replicaCount: 1 - url: "" # only change if pointing to existing redis cluster and enabled: false - password: "budibase" # recommended to override if using built-in redis + # -- If you choose to run Redis externally to this chart, you can specify the + # connection details here. + url: "" + # -- The password to use when connecting to Redis. It's recommended that you change + # this from the default if you're running Redis in-cluster. + password: "budibase" + # -- How much persistent storage to allocate to Redis. storage: 100Mi - ## If defined, storageClassName: - ## If set to "-", storageClassName: "", which disables dynamic provisioning - ## If undefined (the default) or set to null, no storageClassName spec is - ## set, choosing the default provisioner. + # -- If defined, storageClassName: If set to "-", + # storageClassName: "", which disables dynamic provisioning If undefined + # (the default) or set to null, no storageClassName spec is set, choosing + # the default provisioner. storageClass: "" + # -- The resources to use for Redis pods. See + # + # for more information on how to set these. resources: {} objectStore: - # Set to false if using another object store such as S3 + # -- Set to false if using another object store, such as S3. You will need + # to set `services.objectStore.url` to point to your bucket if you do this. minio: true + # -- Whether to enable the Minio web console or not. If you're exposing + # Minio to the Internet (via a custom Ingress record, for example), you + # should set this to false. If you're only exposing Minio to your cluster, + # you can leave this as true. browser: true + # @ignore port: 9000 + # @ignore (you should leave this as 1, we don't support clustering Minio) replicaCount: 1 - accessKey: "" # AWS_ACCESS_KEY if using S3 or existing minio access key - secretKey: "" # AWS_SECRET_ACCESS_KEY if using S3 or existing minio secret - region: "" # AWS_REGION if using S3 or existing minio secret - url: "http://minio-service:9000" # only change if pointing to existing minio cluster or S3 and minio: false + # -- AWS_ACCESS_KEY if using S3 + accessKey: "" + # -- AWS_SECRET_ACCESS_KEY if using S3 + secretKey: "" + # -- AWS_REGION if using S3 + region: "" + # -- URL to use for object storage. Only change this if you're using an + # external object store, such as S3. Remember to set `minio: false` if you + # do this. + url: "http://minio-service:9000" + # -- How much storage to give Minio in its PersistentVolumeClaim. storage: 100Mi - ## If defined, storageClassName: - ## If set to "-", storageClassName: "", which disables dynamic provisioning - ## If undefined (the default) or set to null, no storageClassName spec is - ## set, choosing the default provisioner. + # -- If defined, storageClassName: If set to "-", + # storageClassName: "", which disables dynamic provisioning If undefined + # (the default) or set to null, no storageClassName spec is set, choosing + # the default provisioner. storageClass: "" + # -- The resources to use for Minio pods. See + # + # for more information on how to set these. resources: {} cloudfront: - # Set the url of a distribution to enable cloudfront + # -- Set the url of a distribution to enable cloudfront. cdn: "" - # ID of public key stored in cloudfront + # -- ID of public key stored in cloudfront. publicKeyId: "" - # Base64 encoded private key for the above public key + # -- Base64 encoded private key for the above public key. privateKey64: "" -# Override values in couchDB subchart +# Override values in couchDB subchart. We're only specifying the values we're changing. +# If you want to see all of the available values, see: +# https://github.com/apache/couchdb-helm/tree/couchdb-4.3.0/couchdb couchdb: - ## clusterSize is the initial size of the CouchDB cluster. + # -- The number of replicas to run in the CouchDB cluster. We set this to + # 1 by default to make things simpler, but you can set it to 3 if you need + # a high-availability CouchDB cluster. clusterSize: 1 - allowAdminParty: false - # Secret Management - createAdminSecret: true - - # adminUsername: budibase - # adminPassword: budibase - # adminHash: -pbkdf2-this_is_not_necessarily_secure_either - # cookieAuthSecret: admin - - ## When enabled, will deploy a networkpolicy that allows CouchDB pods to - ## communicate with each other for clustering and ingress on port 5984 - networkPolicy: - enabled: true - - # Use a service account - serviceAccount: - enabled: true - create: true - # name: - # imagePullSecrets: - # - name: myimagepullsecret - - ## The storage volume used by each Pod in the StatefulSet. If a - ## persistentVolume is not enabled, the Pods will use `emptyDir` ephemeral - ## local storage. Setting the storageClass attribute to "-" disables dynamic - ## provisioning of Persistent Volumes; leaving it unset will invoke the default - ## provisioner. - persistentVolume: - enabled: false - accessModes: - - ReadWriteOnce - size: 10Gi - storageClass: "" - - ## The CouchDB image + # -- We use a custom CouchDB image for running Budibase and we don't support + # using any other CouchDB image. You shouldn't change this, and if you do we + # can't guarantee that Budibase will work. image: - repository: couchdb - tag: 3.1.1 - pullPolicy: IfNotPresent - - ## Experimental integration with Lucene-powered fulltext search - enableSearch: true - searchImage: - repository: kocolosk/couchdb-search - tag: 0.2.0 - pullPolicy: IfNotPresent - - initImage: - repository: busybox - tag: latest + # @ignore + repository: budibase/couchdb + # @ignore + tag: v3.2.1 + # @ignore pullPolicy: Always - ## CouchDB is happy to spin up cluster nodes in parallel, but if you encounter - ## problems you can try setting podManagementPolicy to the StatefulSet default - ## `OrderedReady` - podManagementPolicy: Parallel + # @ignore + # This should remain false. We ship Clouseau ourselves as part of the + # budibase/couchdb image, and it's not possible to disable it because it's a + # core part of the Budibase experience. + enableSearch: false - ## Optional pod annotations - annotations: {} - - ## Optional tolerations - tolerations: [] - - affinity: {} - - service: - # annotations: - enabled: true - type: ClusterIP - externalPort: 5984 - - ## An Ingress resource can provide name-based virtual hosting and TLS - ## termination among other things for CouchDB deployments which are accessed - ## from outside the Kubernetes cluster. - ## ref: https://kubernetes.io/docs/concepts/services-networking/ingress/ - ingress: - enabled: false - hosts: - - chart-example.local - path: / - annotations: - [] - # kubernetes.io/ingress.class: nginx - # kubernetes.io/tls-acme: "true" - tls: - # Secrets must be manually created in the namespace. - # - secretName: chart-example-tls - # hosts: - # - chart-example.local - - ## Optional resource requests and limits for the CouchDB container - ## ref: http://kubernetes.io/docs/user-guide/compute-resources/ - resources: - {} - # requests: - # cpu: 100m - # memory: 128Mi - # limits: - # cpu: 56 - # memory: 256Gi - - ## erlangFlags is a map that is passed to the Erlang VM as flags using the - ## ERL_FLAGS env. `name` and `setcookie` flags are minimally required to - ## establish connectivity between cluster nodes. - ## ref: http://erlang.org/doc/man/erl.html#init_flags - erlangFlags: - name: couchdb - setcookie: monster - - ## couchdbConfig will override default CouchDB configuration settings. - ## The contents of this map are reformatted into a .ini file laid down - ## by a ConfigMap object. - ## ref: http://docs.couchdb.org/en/latest/config/index.html couchdbConfig: couchdb: - uuid: budibase-couchdb # REQUIRED: Unique identifier for this CouchDB server instance - # cluster: - # q: 8 # Create 8 shards for each database - chttpd: - bind_address: any - # chttpd.require_valid_user disables all the anonymous requests to the port - # 5984 when is set to true. - require_valid_user: false - - # Kubernetes local cluster domain. - # This is used to generate FQDNs for peers when joining the CouchDB cluster. - dns: - clusterDomainSuffix: cluster.local - - ## Configure liveness and readiness probe values - ## Ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes - # FOR COUCHDB - livenessProbe: - failureThreshold: 3 - initialDelaySeconds: 0 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 - readinessProbe: - failureThreshold: 3 - initialDelaySeconds: 0 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 + # -- Unique identifier for this CouchDB server instance. You shouldn't need + # to change this. + uuid: budibase-couchdb diff --git a/hosting/couchdb/runner.sh b/hosting/couchdb/runner.sh index b576c886c2..e56b8e0e7f 100644 --- a/hosting/couchdb/runner.sh +++ b/hosting/couchdb/runner.sh @@ -30,10 +30,18 @@ elif [[ "${TARGETBUILD}" = "single" ]]; then # mount, so we use that for all persistent data. sed -i "s#DATA_DIR#/data#g" /opt/clouseau/clouseau.ini sed -i "s#DATA_DIR#/data#g" /opt/couchdb/etc/local.ini +elif [[ "${TARGETBUILD}" = "docker-compose" ]]; then + # We remove the database_dir and view_index_dir settings from the local.ini + # in docker-compose because it will default to /opt/couchdb/data which is what + # our docker-compose was using prior to us switching to using our own CouchDB + # image. + sed -i "s#^database_dir.*\$##g" /opt/couchdb/etc/local.ini + sed -i "s#^view_index_dir.*\$##g" /opt/couchdb/etc/local.ini + sed -i "s#^dir=.*\$#dir=/opt/couchdb/data#g" /opt/clouseau/clouseau.ini elif [[ -n $KUBERNETES_SERVICE_HOST ]]; then # In Kubernetes the directory /opt/couchdb/data has a persistent volume # mount for storing database data. - sed -i "s#DATA_DIR#/opt/couchdb/data#g" /opt/clouseau/clouseau.ini + sed -i "s#^dir=.*\$#dir=/opt/couchdb/data#g" /opt/clouseau/clouseau.ini # We remove the database_dir and view_index_dir settings from the local.ini # in Kubernetes because it will default to /opt/couchdb/data which is what diff --git a/hosting/docker-compose.yaml b/hosting/docker-compose.yaml index 8f66d211f7..7803916069 100644 --- a/hosting/docker-compose.yaml +++ b/hosting/docker-compose.yaml @@ -57,7 +57,6 @@ services: depends_on: - redis-service - minio-service - - couch-init minio-service: restart: unless-stopped @@ -70,7 +69,7 @@ services: MINIO_BROWSER: "off" command: server /data --console-address ":9001" healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] + test: "timeout 5s bash -c ':> /dev/tcp/127.0.0.1/9000' || exit 1" interval: 30s timeout: 20s retries: 3 @@ -98,26 +97,15 @@ services: couchdb-service: restart: unless-stopped - image: ibmcom/couchdb3 + image: budibase/couchdb + pull_policy: always environment: - COUCHDB_PASSWORD=${COUCH_DB_PASSWORD} - COUCHDB_USER=${COUCH_DB_USER} + - TARGETBUILD=docker-compose volumes: - couchdb3_data:/opt/couchdb/data - couch-init: - image: curlimages/curl - environment: - PUT_CALL: "curl -u ${COUCH_DB_USER}:${COUCH_DB_PASSWORD} -X PUT couchdb-service:5984" - depends_on: - - couchdb-service - command: - [ - "sh", - "-c", - "sleep 10 && $${PUT_CALL}/_users && $${PUT_CALL}/_replicator; fg;", - ] - redis-service: restart: unless-stopped image: redis diff --git a/hosting/nginx.dev.conf b/hosting/nginx.dev.conf index 915125cbce..f0a58a9a98 100644 --- a/hosting/nginx.dev.conf +++ b/hosting/nginx.dev.conf @@ -42,7 +42,7 @@ http { server { listen 10000 default_server; server_name _; - client_max_body_size 1000m; + client_max_body_size 50000m; ignore_invalid_headers off; proxy_buffering off; diff --git a/hosting/proxy/nginx.prod.conf b/hosting/proxy/nginx.prod.conf index 6da2e4a1c3..88f9645f80 100644 --- a/hosting/proxy/nginx.prod.conf +++ b/hosting/proxy/nginx.prod.conf @@ -249,4 +249,30 @@ http { gzip_comp_level 6; gzip_types text/plain text/css text/xml application/json application/javascript application/rss+xml application/atom+xml image/svg+xml; } + + # From https://docs.datadoghq.com/integrations/nginx/?tab=kubernetes + server { + listen 81; + server_name localhost; + + access_log off; + allow 127.0.0.1; + deny all; + + location /nginx_status { + # Choose your status module + + # freely available with open source NGINX + stub_status; + + # for open source NGINX < version 1.7.5 + # stub_status on; + + # available only with NGINX Plus + # status; + + # ensures the version information can be retrieved + server_tokens on; + } + } } diff --git a/lerna.json b/lerna.json index 3540818872..016a781732 100644 --- a/lerna.json +++ b/lerna.json @@ -1,5 +1,5 @@ { - "version": "2.13.30", + "version": "2.13.35", "npmClient": "yarn", "packages": [ "packages/*", diff --git a/package.json b/package.json index 61a6b187d0..3aa3f7c15d 100644 --- a/package.json +++ b/package.json @@ -17,14 +17,14 @@ "kill-port": "^1.6.1", "lerna": "7.1.1", "madge": "^6.0.0", - "minimist": "^1.2.8", "nx": "16.4.3", "nx-cloud": "16.0.5", "prettier": "2.8.8", "prettier-plugin-svelte": "^2.3.0", "svelte": "3.49.0", "svelte-eslint-parser": "^0.33.1", - "typescript": "5.2.2" + "typescript": "5.2.2", + "yargs": "^17.7.2" }, "scripts": { "preinstall": "node scripts/syncProPackage.js", @@ -82,7 +82,8 @@ "security:audit": "node scripts/audit.js", "postinstall": "husky install", "submodules:load": "git submodule init && git submodule update && yarn", - "submodules:unload": "git submodule deinit --all && yarn" + "submodules:unload": "git submodule deinit --all && yarn", + "add-app-migration": "node scripts/add-app-migration.js --title" }, "workspaces": { "packages": [ diff --git a/packages/backend-core/src/auth/auth.ts b/packages/backend-core/src/auth/auth.ts index e31bc81eed..1951c7986c 100644 --- a/packages/backend-core/src/auth/auth.ts +++ b/packages/backend-core/src/auth/auth.ts @@ -19,6 +19,7 @@ import { GoogleInnerConfig, OIDCInnerConfig, PlatformLogoutOpts, + SessionCookie, SSOProviderType, } from "@budibase/types" import * as events from "../events" @@ -44,7 +45,6 @@ export const buildAuthMiddleware = authenticated export const buildTenancyMiddleware = tenancy export const buildCsrfMiddleware = csrf export const passport = _passport -export const jwt = require("jsonwebtoken") // Strategies _passport.use(new LocalStrategy(local.options, local.authenticate)) @@ -191,10 +191,10 @@ export async function platformLogout(opts: PlatformLogoutOpts) { if (!ctx) throw new Error("Koa context must be supplied to logout.") - const currentSession = getCookie(ctx, Cookie.Auth) + const currentSession = getCookie(ctx, Cookie.Auth) let sessions = await getSessionsForUser(userId) - if (keepActiveSession) { + if (currentSession && keepActiveSession) { sessions = sessions.filter( session => session.sessionId !== currentSession.sessionId ) diff --git a/packages/backend-core/src/middleware/authenticated.ts b/packages/backend-core/src/middleware/authenticated.ts index 8bd6591d05..16f658b90a 100644 --- a/packages/backend-core/src/middleware/authenticated.ts +++ b/packages/backend-core/src/middleware/authenticated.ts @@ -13,7 +13,7 @@ import { getGlobalDB, doInTenant } from "../context" import { decrypt } from "../security/encryption" import * as identity from "../context/identity" import env from "../environment" -import { Ctx, EndpointMatcher } from "@budibase/types" +import { Ctx, EndpointMatcher, SessionCookie } from "@budibase/types" import { InvalidAPIKeyError, ErrorCode } from "../errors" const ONE_MINUTE = env.SESSION_UPDATE_PERIOD @@ -98,7 +98,9 @@ export default function ( // check the actual user is authenticated first, try header or cookie let headerToken = ctx.request.headers[Header.TOKEN] - const authCookie = getCookie(ctx, Cookie.Auth) || openJwt(headerToken) + const authCookie = + getCookie(ctx, Cookie.Auth) || + openJwt(headerToken) let apiKey = ctx.request.headers[Header.API_KEY] if (!apiKey && ctx.request.headers[Header.AUTHORIZATION]) { diff --git a/packages/backend-core/src/middleware/passport/datasource/google.ts b/packages/backend-core/src/middleware/passport/datasource/google.ts index ae6b3b4913..ab4ffee9d2 100644 --- a/packages/backend-core/src/middleware/passport/datasource/google.ts +++ b/packages/backend-core/src/middleware/passport/datasource/google.ts @@ -3,7 +3,7 @@ import { Cookie } from "../../../constants" import * as configs from "../../../configs" import * as cache from "../../../cache" import * as utils from "../../../utils" -import { UserCtx, SSOProfile } from "@budibase/types" +import { UserCtx, SSOProfile, DatasourceAuthCookie } from "@budibase/types" import { ssoSaveUserNoOp } from "../sso/sso" const GoogleStrategy = require("passport-google-oauth").OAuth2Strategy @@ -58,7 +58,14 @@ export async function postAuth( const platformUrl = await configs.getPlatformUrl({ tenantAware: false }) let callbackUrl = `${platformUrl}/api/global/auth/datasource/google/callback` - const authStateCookie = utils.getCookie(ctx, Cookie.DatasourceAuth) + const authStateCookie = utils.getCookie<{ appId: string }>( + ctx, + Cookie.DatasourceAuth + ) + + if (!authStateCookie) { + throw new Error("Unable to fetch datasource auth cookie") + } return passport.authenticate( new GoogleStrategy( diff --git a/packages/backend-core/src/objectStore/objectStore.ts b/packages/backend-core/src/objectStore/objectStore.ts index 1971c09e9d..9b44eace49 100644 --- a/packages/backend-core/src/objectStore/objectStore.ts +++ b/packages/backend-core/src/objectStore/objectStore.ts @@ -305,20 +305,33 @@ export async function retrieveDirectory(bucketName: string, path: string) { let writePath = join(budibaseTempDir(), v4()) fs.mkdirSync(writePath) const objects = await listAllObjects(bucketName, path) - let fullObjects = await Promise.all( - objects.map(obj => retrieve(bucketName, obj.Key!)) + let streams = await Promise.all( + objects.map(obj => getReadStream(bucketName, obj.Key!)) ) let count = 0 + const writePromises: Promise[] = [] for (let obj of objects) { const filename = obj.Key! - const data = fullObjects[count++] + const stream = streams[count++] const possiblePath = filename.split("/") - if (possiblePath.length > 1) { - const dirs = possiblePath.slice(0, possiblePath.length - 1) - fs.mkdirSync(join(writePath, ...dirs), { recursive: true }) + const dirs = possiblePath.slice(0, possiblePath.length - 1) + const possibleDir = join(writePath, ...dirs) + if (possiblePath.length > 1 && !fs.existsSync(possibleDir)) { + fs.mkdirSync(possibleDir, { recursive: true }) } - fs.writeFileSync(join(writePath, ...possiblePath), data) + const writeStream = fs.createWriteStream(join(writePath, ...possiblePath), { + mode: 0o644, + }) + stream.pipe(writeStream) + writePromises.push( + new Promise((resolve, reject) => { + stream.on("finish", resolve) + stream.on("error", reject) + writeStream.on("error", reject) + }) + ) } + await Promise.all(writePromises) return writePath } diff --git a/packages/backend-core/src/queue/constants.ts b/packages/backend-core/src/queue/constants.ts index e1ffcfee36..eb4f21aced 100644 --- a/packages/backend-core/src/queue/constants.ts +++ b/packages/backend-core/src/queue/constants.ts @@ -3,4 +3,5 @@ export enum JobQueue { APP_BACKUP = "appBackupQueue", AUDIT_LOG = "auditLogQueue", SYSTEM_EVENT_QUEUE = "systemEventQueue", + APP_MIGRATION = "appMigration", } diff --git a/packages/backend-core/src/queue/listeners.ts b/packages/backend-core/src/queue/listeners.ts index 42e3172364..063a01bd2f 100644 --- a/packages/backend-core/src/queue/listeners.ts +++ b/packages/backend-core/src/queue/listeners.ts @@ -87,6 +87,7 @@ enum QueueEventType { APP_BACKUP_EVENT = "app-backup-event", AUDIT_LOG_EVENT = "audit-log-event", SYSTEM_EVENT = "system-event", + APP_MIGRATION = "app-migration", } const EventTypeMap: { [key in JobQueue]: QueueEventType } = { @@ -94,6 +95,7 @@ const EventTypeMap: { [key in JobQueue]: QueueEventType } = { [JobQueue.APP_BACKUP]: QueueEventType.APP_BACKUP_EVENT, [JobQueue.AUDIT_LOG]: QueueEventType.AUDIT_LOG_EVENT, [JobQueue.SYSTEM_EVENT_QUEUE]: QueueEventType.SYSTEM_EVENT, + [JobQueue.APP_MIGRATION]: QueueEventType.APP_MIGRATION, } function logging(queue: Queue, jobQueue: JobQueue) { diff --git a/packages/backend-core/src/redis/redlockImpl.ts b/packages/backend-core/src/redis/redlockImpl.ts index a7b2e2b4c6..4de2516ab2 100644 --- a/packages/backend-core/src/redis/redlockImpl.ts +++ b/packages/backend-core/src/redis/redlockImpl.ts @@ -109,7 +109,7 @@ export async function doWithLock( ): Promise> { const redlock = await getClient(opts.type, opts.customOptions) let lock: Redlock.Lock | undefined - let timeout: NodeJS.Timeout | undefined + let timeout try { const name = getLockName(opts) diff --git a/packages/backend-core/src/security/encryption.ts b/packages/backend-core/src/security/encryption.ts index 7a8cfaf04a..45ed566a92 100644 --- a/packages/backend-core/src/security/encryption.ts +++ b/packages/backend-core/src/security/encryption.ts @@ -73,6 +73,9 @@ export async function encryptFile( const outputFileName = `${filename}.enc` const filePath = join(dir, filename) + if (fs.lstatSync(filePath).isDirectory()) { + throw new Error("Unable to encrypt directory") + } const inputFile = fs.createReadStream(filePath) const outputFile = fs.createWriteStream(join(dir, outputFileName)) @@ -110,6 +113,9 @@ export async function decryptFile( outputPath: string, secret: string ) { + if (fs.lstatSync(inputPath).isDirectory()) { + throw new Error("Unable to encrypt directory") + } const { salt, iv } = await getSaltAndIV(inputPath) const inputFile = fs.createReadStream(inputPath, { start: SALT_LENGTH + IV_LENGTH, diff --git a/packages/backend-core/src/tenancy/tenancy.ts b/packages/backend-core/src/tenancy/tenancy.ts index 7b17bdbe18..3603ef3462 100644 --- a/packages/backend-core/src/tenancy/tenancy.ts +++ b/packages/backend-core/src/tenancy/tenancy.ts @@ -93,11 +93,19 @@ export const getTenantIDFromCtx = ( // subdomain if (isAllowed(TenantResolutionStrategy.SUBDOMAIN)) { // e.g. budibase.app or local.com:10000 - const platformHost = new URL(getPlatformURL()).host.split(":")[0] + let platformHost + try { + platformHost = new URL(getPlatformURL()).host.split(":")[0] + } catch (err: any) { + // if invalid URL, just don't try to process subdomain + if (err.code !== "ERR_INVALID_URL") { + throw err + } + } // e.g. tenant.budibase.app or tenant.local.com const requestHost = ctx.host // parse the tenant id from the difference - if (requestHost.includes(platformHost)) { + if (platformHost && requestHost.includes(platformHost)) { const tenantId = requestHost.substring( 0, requestHost.indexOf(`.${platformHost}`) diff --git a/packages/backend-core/src/utils/utils.ts b/packages/backend-core/src/utils/utils.ts index b10d9ebdc0..ee1ef6da0c 100644 --- a/packages/backend-core/src/utils/utils.ts +++ b/packages/backend-core/src/utils/utils.ts @@ -11,8 +11,7 @@ import { TenantResolutionStrategy, } from "@budibase/types" import type { SetOption } from "cookies" - -const jwt = require("jsonwebtoken") +import jwt, { Secret } from "jsonwebtoken" const APP_PREFIX = DocumentType.APP + SEPARATOR const PROD_APP_PREFIX = "/app/" @@ -60,10 +59,7 @@ export function isServingApp(ctx: Ctx) { return true } // prod app - if (ctx.path.startsWith(PROD_APP_PREFIX)) { - return true - } - return false + return ctx.path.startsWith(PROD_APP_PREFIX) } export function isServingBuilder(ctx: Ctx): boolean { @@ -138,16 +134,16 @@ function parseAppIdFromUrl(url?: string) { * opens the contents of the specified encrypted JWT. * @return the contents of the token. */ -export function openJwt(token: string) { +export function openJwt(token?: string): T | undefined { if (!token) { - return token + return undefined } try { - return jwt.verify(token, env.JWT_SECRET) + return jwt.verify(token, env.JWT_SECRET as Secret) as T } catch (e) { if (env.JWT_SECRET_FALLBACK) { // fallback to enable rotation - return jwt.verify(token, env.JWT_SECRET_FALLBACK) + return jwt.verify(token, env.JWT_SECRET_FALLBACK) as T } else { throw e } @@ -159,13 +155,9 @@ export function isValidInternalAPIKey(apiKey: string) { return true } // fallback to enable rotation - if ( - env.INTERNAL_API_KEY_FALLBACK && - env.INTERNAL_API_KEY_FALLBACK === apiKey - ) { - return true - } - return false + return !!( + env.INTERNAL_API_KEY_FALLBACK && env.INTERNAL_API_KEY_FALLBACK === apiKey + ) } /** @@ -173,14 +165,14 @@ export function isValidInternalAPIKey(apiKey: string) { * @param ctx The request which is to be manipulated. * @param name The name of the cookie to get. */ -export function getCookie(ctx: Ctx, name: string) { +export function getCookie(ctx: Ctx, name: string) { const cookie = ctx.cookies.get(name) if (!cookie) { - return cookie + return undefined } - return openJwt(cookie) + return openJwt(cookie) } /** @@ -197,7 +189,7 @@ export function setCookie( opts = { sign: true } ) { if (value && opts && opts.sign) { - value = jwt.sign(value, env.JWT_SECRET) + value = jwt.sign(value, env.JWT_SECRET as Secret) } const config: SetOption = { diff --git a/packages/bbui/src/DetailSummary/DetailSummary.svelte b/packages/bbui/src/DetailSummary/DetailSummary.svelte index daa9f3f5ca..e5d6fda86b 100644 --- a/packages/bbui/src/DetailSummary/DetailSummary.svelte +++ b/packages/bbui/src/DetailSummary/DetailSummary.svelte @@ -1,20 +1,17 @@ diff --git a/packages/bbui/src/Form/Core/Dropzone.svelte b/packages/bbui/src/Form/Core/Dropzone.svelte index dc89476db2..fa0be630ba 100644 --- a/packages/bbui/src/Form/Core/Dropzone.svelte +++ b/packages/bbui/src/Form/Core/Dropzone.svelte @@ -53,7 +53,7 @@ $: { if (selectedImage?.url) { selectedUrl = selectedImage?.url - } else if (selectedImage) { + } else if (selectedImage && isImage) { try { let reader = new FileReader() reader.readAsDataURL(selectedImage) diff --git a/packages/builder/src/builderStore/index.js b/packages/builder/src/builderStore/index.js index ece17cb46f..dd54dcf13e 100644 --- a/packages/builder/src/builderStore/index.js +++ b/packages/builder/src/builderStore/index.js @@ -8,6 +8,7 @@ import { derived, get } from "svelte/store" import { findComponent, findComponentPath } from "./componentUtils" import { RoleUtils } from "@budibase/frontend-core" import { createHistoryStore } from "builderStore/store/history" +import { cloneDeep } from "lodash/fp" export const store = getFrontendStore() export const automationStore = getAutomationStore() @@ -69,7 +70,14 @@ export const selectedComponent = derived( if (!$selectedScreen || !$store.selectedComponentId) { return null } - return findComponent($selectedScreen?.props, $store.selectedComponentId) + const selected = findComponent( + $selectedScreen?.props, + $store.selectedComponentId + ) + + const clone = selected ? cloneDeep(selected) : selected + store.actions.components.migrateSettings(clone) + return clone } ) diff --git a/packages/builder/src/builderStore/store/frontend.js b/packages/builder/src/builderStore/store/frontend.js index a4729b4a8a..aaa0eb0184 100644 --- a/packages/builder/src/builderStore/store/frontend.js +++ b/packages/builder/src/builderStore/store/frontend.js @@ -85,6 +85,7 @@ const INITIAL_FRONTEND_STATE = { selectedScreenId: null, selectedComponentId: null, selectedLayoutId: null, + hoverComponentId: null, // Client state selectedComponentInstance: null, @@ -112,7 +113,7 @@ export const getFrontendStore = () => { } let clone = cloneDeep(screen) const result = patchFn(clone) - + // An explicit false result means skip this change if (result === false) { return } @@ -601,6 +602,36 @@ export const getFrontendStore = () => { // Finally try an external table return validTables.find(table => table.sourceType === DB_TYPE_EXTERNAL) }, + migrateSettings: enrichedComponent => { + const componentPrefix = "@budibase/standard-components" + let migrated = false + + if (enrichedComponent?._component == `${componentPrefix}/formblock`) { + // Use default config if the 'buttons' prop has never been initialised + if (!("buttons" in enrichedComponent)) { + enrichedComponent["buttons"] = + Utils.buildDynamicButtonConfig(enrichedComponent) + migrated = true + } else if (enrichedComponent["buttons"] == null) { + // Ignore legacy config if 'buttons' has been reset by 'resetOn' + const { _id, actionType, dataSource } = enrichedComponent + enrichedComponent["buttons"] = Utils.buildDynamicButtonConfig({ + _id, + actionType, + dataSource, + }) + migrated = true + } + + // Ensure existing Formblocks position their buttons at the top. + if (!("buttonPosition" in enrichedComponent)) { + enrichedComponent["buttonPosition"] = "top" + migrated = true + } + } + + return migrated + }, enrichEmptySettings: (component, opts) => { if (!component?._component) { return @@ -672,7 +703,6 @@ export const getFrontendStore = () => { component[setting.key] = setting.defaultValue } } - // Validate non-empty settings else { if (setting.type === "dataProvider") { @@ -722,6 +752,9 @@ export const getFrontendStore = () => { useDefaultValues: true, }) + // Migrate nested component settings + store.actions.components.migrateSettings(instance) + // Add any extra properties the component needs let extras = {} if (definition.hasChildren) { @@ -845,7 +878,16 @@ export const getFrontendStore = () => { if (!component) { return false } - return patchFn(component, screen) + + // Mutates the fetched component with updates + const patchResult = patchFn(component, screen) + + // Mutates the component with any required settings updates + const migrated = store.actions.components.migrateSettings(component) + + // Returning an explicit false signifies that we should skip this + // update. If we migrated something, ensure we never skip. + return migrated ? null : patchResult } await store.actions.screens.patch(patchScreen, screenId) }, @@ -1247,9 +1289,13 @@ export const getFrontendStore = () => { const settings = getComponentSettings(component._component) const updatedSetting = settings.find(setting => setting.key === name) - const resetFields = settings.filter( - setting => name === setting.resetOn - ) + // Can be a single string or array of strings + const resetFields = settings.filter(setting => { + return ( + name === setting.resetOn || + (Array.isArray(setting.resetOn) && setting.resetOn.includes(name)) + ) + }) resetFields?.forEach(setting => { component[setting.key] = null }) @@ -1271,6 +1317,7 @@ export const getFrontendStore = () => { }) } component[name] = value + return true } }, requestEjectBlock: componentId => { @@ -1278,7 +1325,6 @@ export const getFrontendStore = () => { }, handleEjectBlock: async (componentId, ejectedDefinition) => { let nextSelectedComponentId - await store.actions.screens.patch(screen => { const block = findComponent(screen.props, componentId) const parent = findComponentParent(screen.props, componentId) diff --git a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowChart.svelte b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowChart.svelte index 84fa62fe69..ffe769d024 100644 --- a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowChart.svelte +++ b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowChart.svelte @@ -57,16 +57,11 @@ }} class="buttons" > - +
Run test
- +
{ diff --git a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItemHeader.svelte b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItemHeader.svelte index 3c9e1a13b1..5ddba31bb8 100644 --- a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItemHeader.svelte +++ b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItemHeader.svelte @@ -97,6 +97,7 @@ class:typing={typing && !automationNameError} class:typing-error={automationNameError} class="blockSection" + on:click={() => dispatch("toggle")} >
@@ -138,7 +139,20 @@ on:input={e => { automationName = e.target.value.trim() }} - on:click={startTyping} + on:click={e => { + e.stopPropagation() + startTyping() + }} + on:keydown={async e => { + if (e.key === "Enter") { + typing = false + if (automationNameError) { + automationName = stepNames[block.id] || block?.name + } else { + await saveName() + } + } + }} on:blur={async () => { typing = false if (automationNameError) { @@ -168,7 +182,11 @@
dispatch("toggle")} + e.stopPropagation() + on:click={e => { + e.stopPropagation() + dispatch("toggle") + }} hoverable name={open ? "ChevronUp" : "ChevronDown"} /> @@ -195,7 +213,10 @@ {/if} {#if !showTestStatus} dispatch("toggle")} + on:click={e => { + e.stopPropagation() + dispatch("toggle") + }} hoverable name={open ? "ChevronUp" : "ChevronDown"} /> diff --git a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/TestDataModal.svelte b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/TestDataModal.svelte index 5c97d77ae8..76def72bf6 100644 --- a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/TestDataModal.svelte +++ b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/TestDataModal.svelte @@ -1,11 +1,9 @@ - - -
- -
- -
- -
-