diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 696bd18986..7abfe537e9 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -2,7 +2,7 @@ From opening a bug report to creating a pull request: every contribution is appreciated and welcome. If you're planning to implement a new feature or change the api please create an issue first. This way we can ensure that your precious work is not in vain. -### Not Sure Where to Start? +## Not Sure Where to Start? Budibase is a low-code web application builder that creates svelte based web applications. @@ -14,6 +14,8 @@ Budibase is a monorepo managed by [lerna](https://github.com/lerna/lerna). Lerna - **packages/server** - The budibase server. This [Koa](https://koajs.com/) app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system. +- **packages/worker** - This [Koa](https://koajs.com/) app is responsible for providing global apis for managing your budibase installation. Authentication, Users, Email, Org and Auth configs are all provided by the worker. + ## Contributor License Agreement (CLA) In order to accept your pull request, we need you to submit a CLA. You only need to do this once. If you are submitting a pull request for the first time, just submit a Pull Request and our CLA Bot will give you instructions on how to sign the CLA before merging your Pull Request. @@ -62,8 +64,6 @@ A component is the basic frontend building block of a budibase app. Component libraries are collections of components as well as the definition of their props contained in a file called `components.json`. - - ## Contributing to Budibase * Please maintain the existing code style. @@ -74,31 +74,30 @@ Component libraries are collections of components as well as the definition of t * If the project diverges from your branch, please rebase instead of merging. This makes the commit graph easier to read. -* Once your work is completed, please raise a PR against the main branch with some information about what has changed and why. +* Once your work is completed, please raise a PR against the `develop` branch with some information about what has changed and why. ### Getting Started For Contributors - -### 1. Prerequisites +#### 1. Prerequisites *yarn -* `npm install -g yarn` *jest* - `npm install -g jest` -### 2. Clone this repository +#### 2. Clone this repository `git clone https://github.com/Budibase/budibase.git` then `cd ` into your local copy. -### 3. Install and Build +#### 3. Install and Build To develop the Budibase platform you'll need [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) installed. -#### Quick method +##### Quick method `yarn setup` will check that all necessary components are installed and setup the repo for usage. -#### Manual method +##### Manual method The following commands can be executed to manually get Budibase up and running (assuming Docker/Docker Compose has been installed). @@ -108,15 +107,7 @@ The following commands can be executed to manually get Budibase up and running ( `yarn build` will build all budibase packages. -### 4. Initialising Budibase and Creating a Budibase App - -Starting up the budibase electron app should initialise budibase for you. A Budibase apps folder will have been created in `~/.budibase`. - -This is a blank apps folder, so you will need to create yourself an app, you can do this by clicking "Create New App" from the budibase builder. - -This will create a new budibase application in the `~/.budibase/` directory, and NPM install the component libraries for that application. Let's start building your app with the budibase builder! - -### 4. Running +#### 4. Running To run the budibase server and builder in dev mode (i.e. with live reloading): @@ -126,7 +117,7 @@ To run the budibase server and builder in dev mode (i.e. with live reloading): This will enable watch mode for both the builder app, server, client library and any component libraries. -### 5. Debugging using VS Code +#### 5. Debugging using VS Code To debug the budibase server and worker a VS Code launch configuration has been provided. @@ -135,75 +126,90 @@ Alternatively to start both components simultaneously select `Start Budibase`. In addition to the above, the remaining budibase components may be ran in dev mode using: `yarn dev:noserver`. -### 6. Cleanup +#### 6. Cleanup If you wish to delete all the apps created in development and reset the environment then run the following: 1. `yarn nuke:docker` will wipe all the Budibase services 2. `yarn dev` will restart all the services -## Data Storage - -When you are running locally, budibase stores data on disk using [PouchDB](https://pouchdb.com/), as well as some JSON on local files. After setting up budibase, you can find all of this data in the `~/.budibase` directory. - -A client can have one or more budibase applications. Budibase applications are stored in `~/.budibase/`. Files used by your budibase application when running are stored in the `public` directory. Everything else is dev files used for the development of your apps in the builder. - -#### Frontend - -To see the current individual JSON definitions for your pages and screens used by the builder, have a look at `~/.budibase//pages`. - -For your actual running application (not in dev), the frontend tree structure of the application (known as `clientFrontendDefinition`) is stored as JSON on disk. This is what the budibase client library reads to create your app at runtime. This can be found at `~/.budibase//public/clientFrontendDefinition.js` - -The HTML and CSS for your apps runtime pages, as well as the budibase client library JS is stored at: - -- `~/.budibase//public/main` -- `~/.budibase//public/unauthenticated` - -#### Backend +### Backend For the backend we run [Redis](https://redis.io/), [CouchDB](https://couchdb.apache.org/), [MinIO](https://min.io/) and [Envoy](https://www.envoyproxy.io/) in Docker compose. This means that to develop Budibase you will need Docker and Docker compose installed. The backend services are then ran separately as Node services with nodemon so that they can be debugged outside of Docker. -### Publishing Budibase to NPM +### Data Storage -#### Testing In Electron +When you are running locally, budibase stores data on disk using docker volumes. The volumes and the types of data associated with each are: -At budibase, we pride ourselves on giving our users a fast, native and slick local development experience. As a result, we use the electron to provide a native GUI for the budibase builder. In order to release budibase out into the wild, you should test your changes in a packaged electron application. To do this, first build budibase from the root directory. +- `redis_data` + - Sessions, email tokens +- `couchdb3_data` + - Global and app databases +- `minio_data` + - App manifest, budibase client, static assets + +### Devlopment Modes + +A combination of environment variables controls the mode that budibase runs in. +Yarn commands can be used to mimic the different modes that budibase can be ran in + +#### Self Hosted +The default mode. A single tenant installation with no usage restrictions. + +To enable this mode, use: ``` -yarn build +yarn mode:self ``` -Now everything is built, you can package up your electron application. +#### Cloud +The cloud mode, with account portal turned off. + +To enable this mode, use: ``` -cd packages/server -yarn build:electron +yarn mode:cloud ``` - -Your new electron application will be stored in `packages/server/dist/`. Open up the executable and make sure everything is working smoothly. +#### Cloud & Account +The cloud mode, with account portal turned on. This is a replica of the mode that runs at https://budibase.app -#### Publishing to NPM - -Once you are happy that your changes work in electron, you can publish all the latest versions of the monorepo packages by running: - +To enable this mode, use: ``` -yarn publishnpm +yarn mode:account ``` +### CI -from your root directory. +#### PR Job -#### CI Release +After your pr is submitted a github action (can be found at `.github/workflows/budibase_ci.yml`) will run to perform some checks against the changes such as linting, build and test. -After NPM has successfully published the budibase packages, a new tag will be pushed to master. This will kick off a github action (can be found at `.github/workflows/release.yml`) this will build and package the electron application for every OS (Windows, Mac, Linux). The binaries will be stored under the new tag on the [budibase releases page](https://github.com/Budibase/budibase/releases). +The job will run when changes are pushed to or targetted at `master` and `develop` +#### Release Develop + +To test changes before a release, a prerelease action (can be found at `.github/workflows/release-develop.yml`) will run to build and release develop versions of npm packages and docker images. On each subsequent commit to develop a new alpha version of npm packages will be created and released. + +For example: + +- `feature1` -> `develop` = `v0.9.160-alpha.1` +- `feature2` -> `develop` = `v0.9.160-alpha.0` + +The job will run when changes are pushed to `develop` +#### Release Job + +To release changes a release job (can be found at `.github/workflows/release.yml`) will run to create final versions of npm packages and docker images. + +Following the example above: + +- `develop` -> `master` = `v0.9.160` + +The job will run when changes are pushed to `master` + +#### Release Self Host Job + +To release the self hosted version of docker images, an additional job (can be found at `.github/workflows/release-selfhost.yml`) must be ran manually. This will releaae docker images to docker hub under the tag `latest` to be picked up by self hosted installations. ### Troubleshooting -Sometimes, things go wrong. This can be due to incompatible updates on the budibase platform. To clear down your development environment and start again: - -``` -rm -rf ~/.budibase -``` -Follow from **Step 3. Install and Build** in the setup guide above. You should have a fresh Budibase installation. - +Sometimes, things go wrong. This can be due to incompatible updates on the budibase platform. To clear down your development environment and start again follow **Step 6. Cleanup**, then proceed from **Step 3. Install and Build** in the setup guide above. You should have a fresh Budibase installation. ### Running tests #### End-to-end Tests diff --git a/.github/workflows/release-develop.yml b/.github/workflows/release-develop.yml index cf0d6f848c..9389c5bd66 100644 --- a/.github/workflows/release-develop.yml +++ b/.github/workflows/release-develop.yml @@ -9,7 +9,7 @@ env: POSTHOG_TOKEN: ${{ secrets.POSTHOG_TOKEN }} INTERCOM_TOKEN: ${{ secrets.INTERCOM_TOKEN }} POSTHOG_URL: ${{ secrets.POSTHOG_URL }} - SENTRY_DSN: ${{ secrets.SENTRY_DSN }} + SENTRY_DSN: ${{ secrets.SENTRY_DSN_SELF_HOST }} jobs: release: diff --git a/.github/workflows/release-selfhost.yml b/.github/workflows/release-selfhost.yml index a4cbc4e70a..14283aff8f 100644 --- a/.github/workflows/release-selfhost.yml +++ b/.github/workflows/release-selfhost.yml @@ -1,4 +1,4 @@ -name: Budibase Release +name: Budibase Release Selfhost on: workflow_dispatch: @@ -7,7 +7,7 @@ env: POSTHOG_TOKEN: ${{ secrets.POSTHOG_TOKEN }} INTERCOM_TOKEN: ${{ secrets.INTERCOM_TOKEN }} POSTHOG_URL: ${{ secrets.POSTHOG_URL }} - SENTRY_DSN: ${{ secrets.SENTRY_DSN }} + SENTRY_DSN: ${{ secrets.SENTRY_DSN_SELF_HOST }} jobs: release: @@ -28,12 +28,7 @@ jobs: aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: eu-west-1 - - name: 'Get Previous tag' - id: previoustag - uses: "WyriHaximus/github-action-get-previous-tag@v1" - - name: Build/release Docker images (Self Host) - if: ${{ github.event.inputs.release_self_host == 'Y' }} run: | docker login -u $DOCKER_USER -p $DOCKER_PASSWORD yarn build @@ -41,7 +36,7 @@ jobs: env: DOCKER_USER: ${{ secrets.DOCKER_USERNAME }} DOCKER_PASSWORD: ${{ secrets.DOCKER_API_KEY }} - BUDIBASE_RELEASE_VERSION: ${{ steps.previoustag.outputs.tag }} + BUDIBASE_RELEASE_VERSION: latest - uses: azure/setup-helm@v1 id: install diff --git a/docs/budibase-0.2.0.tgz b/docs/budibase-0.2.0.tgz new file mode 100644 index 0000000000..379b92cbb7 Binary files /dev/null and b/docs/budibase-0.2.0.tgz differ diff --git a/docs/index.yaml b/docs/index.yaml index 4e064f3dd0..b050330477 100644 --- a/docs/index.yaml +++ b/docs/index.yaml @@ -1,9 +1,35 @@ apiVersion: v1 entries: budibase: + - apiVersion: v2 + appVersion: 0.9.163 + created: "2021-10-12T21:58:00.515555+01:00" + dependencies: + - condition: services.couchdb.enabled + name: couchdb + repository: https://apache.github.io/couchdb-helm + version: 3.3.4 + - condition: ingress.nginx + name: ingress-nginx + repository: https://github.com/kubernetes/ingress-nginx + version: 3.35.0 + description: Budibase is an open source low-code platform, helping thousands of teams build apps for their workplace in minutes. + digest: f369536c0eac1f6959d51e8ce6d74a87a7a9df29ae84fb9cbed0a273ab77429b + keywords: + - low-code + - database + - cluster + name: budibase + sources: + - https://github.com/Budibase/budibase + - https://budibase.com + type: application + urls: + - https://budibase.github.io/budibase/budibase-0.2.0.tgz + version: 0.2.0 - apiVersion: v2 appVersion: 0.9.56 - created: "2021-08-18T18:41:52.640176+01:00" + created: "2021-10-12T21:58:00.512062+01:00" dependencies: - condition: services.couchdb.enabled name: couchdb @@ -28,7 +54,7 @@ entries: version: 0.1.1 - apiVersion: v2 appVersion: 0.9.56 - created: "2021-08-18T18:41:52.635603+01:00" + created: "2021-10-12T21:58:00.507257+01:00" dependencies: - condition: services.couchdb.enabled name: couchdb @@ -51,4 +77,4 @@ entries: urls: - https://budibase.github.io/budibase/budibase-0.1.0.tgz version: 0.1.0 -generated: "2021-08-18T18:41:52.629415+01:00" +generated: "2021-10-12T21:58:00.503447+01:00" diff --git a/hosting/docker-compose.yaml b/hosting/docker-compose.yaml index 18b93fdf61..91b5a7e32a 100644 --- a/hosting/docker-compose.yaml +++ b/hosting/docker-compose.yaml @@ -21,7 +21,7 @@ services: PORT: 4002 JWT_SECRET: ${JWT_SECRET} LOG_LEVEL: info - SENTRY_DSN: https://a34ae347621946bf8acded18e5b7d4b8@o420233.ingest.sentry.io/5338131 + SENTRY_DSN: https://cc54bb0358fd4300ae97ef2273fbaf9f@o420233.ingest.sentry.io/6007553 ENABLE_ANALYTICS: "true" REDIS_URL: redis-service:6379 REDIS_PASSWORD: ${REDIS_PASSWORD} @@ -51,7 +51,6 @@ services: INTERNAL_API_KEY: ${INTERNAL_API_KEY} REDIS_URL: redis-service:6379 REDIS_PASSWORD: ${REDIS_PASSWORD} - ACCOUNT_PORTAL_URL: https://portal.budi.live volumes: - ./logs:/logs depends_on: diff --git a/hosting/kubernetes/budibase/templates/.helmignore b/hosting/kubernetes/budibase/.helmignore similarity index 100% rename from hosting/kubernetes/budibase/templates/.helmignore rename to hosting/kubernetes/budibase/.helmignore diff --git a/hosting/kubernetes/budibase/Chart.yaml b/hosting/kubernetes/budibase/Chart.yaml index d00b228b0e..fa652ed28f 100644 --- a/hosting/kubernetes/budibase/Chart.yaml +++ b/hosting/kubernetes/budibase/Chart.yaml @@ -22,13 +22,13 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.1.1 +version: 0.2.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. # It is recommended to use it with quotes. -appVersion: "0.9.56" +appVersion: "0.9.163" dependencies: - name: couchdb @@ -37,5 +37,5 @@ dependencies: condition: services.couchdb.enabled - name: ingress-nginx version: 3.35.0 - repository: https://kubernetes.github.io/ingress-nginx - condition: services.ingress.nginx + repository: https://github.com/kubernetes/ingress-nginx + condition: ingress.nginx diff --git a/hosting/kubernetes/budibase/templates/alb-ingress.yaml b/hosting/kubernetes/budibase/templates/alb-ingress.yaml index ea3bd674d5..388bcf1d3e 100644 --- a/hosting/kubernetes/budibase/templates/alb-ingress.yaml +++ b/hosting/kubernetes/budibase/templates/alb-ingress.yaml @@ -7,6 +7,8 @@ metadata: kubernetes.io/ingress.class: alb alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: ip + alb.ingress.kubernetes.io/success-codes: 200,301 + alb.ingress.kubernetes.io/healthcheck-path: / {{- if .Values.ingress.certificateArn }} alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}' alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]' diff --git a/hosting/kubernetes/budibase/templates/app-service-deployment.yaml b/hosting/kubernetes/budibase/templates/app-service-deployment.yaml index 98fdc8dfd0..bce532016a 100644 --- a/hosting/kubernetes/budibase/templates/app-service-deployment.yaml +++ b/hosting/kubernetes/budibase/templates/app-service-deployment.yaml @@ -14,7 +14,7 @@ spec: matchLabels: io.kompose.service: app-service strategy: - type: Recreate + type: RollingUpdate template: metadata: annotations: @@ -73,13 +73,11 @@ spec: name: {{ template "budibase.fullname" . }} key: objectStoreSecret - name: MINIO_URL - {{ if .Values.services.objectStore.url }} value: {{ .Values.services.objectStore.url }} - {{ else }} - value: http://minio-service:{{ .Values.services.objectStore.port }} - {{ end }} - name: PORT value: {{ .Values.services.apps.port | quote }} + - name: MULTI_TENANCY + value: "1" - name: REDIS_PASSWORD value: {{ .Values.services.redis.password }} - name: REDIS_URL @@ -92,14 +90,20 @@ spec: value: {{ .Values.globals.selfHosted | quote }} - name: SENTRY_DSN value: {{ .Values.globals.sentryDSN }} + - name: POSTHOG_TOKEN + value: {{ .Values.globals.posthogToken }} - name: WORKER_URL - value: worker-service:{{ .Values.services.worker.port }} - - name: COOKIE_DOMAIN - value: {{ .Values.globals.cookieDomain | quote }} + value: http://worker-service:{{ .Values.services.worker.port }} + - name: PLATFORM_URL + value: {{ .Values.globals.platformUrl | quote }} + - name: USE_QUOTAS + value: "1" - name: ACCOUNT_PORTAL_URL value: {{ .Values.globals.accountPortalUrl | quote }} - name: ACCOUNT_PORTAL_API_KEY value: {{ .Values.globals.accountPortalApiKey | quote }} + - name: COOKIE_DOMAIN + value: {{ .Values.globals.cookieDomain | quote }} image: budibase/apps imagePullPolicy: Always name: bbapps diff --git a/hosting/kubernetes/budibase/templates/proxy-service-deployment.yaml b/hosting/kubernetes/budibase/templates/proxy-service-deployment.yaml index 0f802da843..2e453d1c5b 100644 --- a/hosting/kubernetes/budibase/templates/proxy-service-deployment.yaml +++ b/hosting/kubernetes/budibase/templates/proxy-service-deployment.yaml @@ -14,7 +14,7 @@ spec: matchLabels: app.kubernetes.io/name: budibase-proxy strategy: - type: Recreate + type: RollingUpdate template: metadata: annotations: @@ -26,7 +26,7 @@ spec: spec: containers: - image: budibase/proxy - imagePullPolicy: "" + imagePullPolicy: Always name: proxy-service ports: - containerPort: {{ .Values.services.proxy.port }} diff --git a/hosting/kubernetes/budibase/templates/worker-service-deployment.yaml b/hosting/kubernetes/budibase/templates/worker-service-deployment.yaml index 08b40d3b6b..563b1b4193 100644 --- a/hosting/kubernetes/budibase/templates/worker-service-deployment.yaml +++ b/hosting/kubernetes/budibase/templates/worker-service-deployment.yaml @@ -15,7 +15,7 @@ spec: matchLabels: io.kompose.service: worker-service strategy: - type: Recreate + type: RollingUpdate template: metadata: annotations: @@ -70,13 +70,11 @@ spec: name: {{ template "budibase.fullname" . }} key: objectStoreSecret - name: MINIO_URL - {{ if .Values.services.objectStore.url }} value: {{ .Values.services.objectStore.url }} - {{ else }} - value: http://minio-service:{{ .Values.services.objectStore.port }} - {{ end }} - name: PORT value: {{ .Values.services.worker.port | quote }} + - name: MULTI_TENANCY + value: "1" - name: REDIS_PASSWORD value: {{ .Values.services.redis.password | quote }} - name: REDIS_URL @@ -91,8 +89,22 @@ spec: value: {{ .Values.globals.accountPortalUrl | quote }} - name: ACCOUNT_PORTAL_API_KEY value: {{ .Values.globals.accountPortalApiKey | quote }} + - name: PLATFORM_URL + value: {{ .Values.globals.platformUrl | quote }} - name: COOKIE_DOMAIN value: {{ .Values.globals.cookieDomain | quote }} + - name: SMTP_FALLBACK_ENABLED + value: {{ .Values.globals.smtp.enabled | quote }} + - name: SMTP_USER + value: {{ .Values.globals.smtp.user | quote }} + - name: SMTP_PASSWORD + value: {{ .Values.globals.smtp.password | quote }} + - name: SMTP_HOST + value: {{ .Values.globals.smtp.host | quote }} + - name: SMTP_PORT + value: {{ .Values.globals.smtp.port | quote }} + - name: SMTP_FROM_ADDRESS + value: {{ .Values.globals.smtp.from | quote }} image: budibase/worker imagePullPolicy: Always name: bbworker diff --git a/hosting/kubernetes/budibase/values.yaml b/hosting/kubernetes/budibase/values.yaml index 5999f9c4bc..774f7ea102 100644 --- a/hosting/kubernetes/budibase/values.yaml +++ b/hosting/kubernetes/budibase/values.yaml @@ -40,11 +40,12 @@ service: port: 10000 ingress: - enabled: true - nginx: true - certificateArn: "" + enabled: false + aws: false + nginx: true + certificateArn: "" className: "" - annotations: + annotations: kubernetes.io/ingress.class: nginx hosts: - host: # change if using custom domain @@ -55,7 +56,7 @@ ingress: service: name: proxy-service port: - number: 10000 + number: 10000 resources: {} # We usually recommend not to specify default resources and to leave this as a conscious @@ -84,20 +85,24 @@ affinity: {} globals: budibaseEnv: PRODUCTION - enableAnalytics: false - posthogToken: "" + enableAnalytics: true sentryDSN: "" + posthogToken: "" logLevel: info - selfHosted: 1 - accountPortalUrL: "" + selfHosted: "" + accountPortalUrl: "" accountPortalApiKey: "" - cookieDomain: "" + cookieDomain: "" + platformUrl: "" + createSecrets: true # creates an internal API key, JWT secrets and redis password for you # if createSecrets is set to false, you can hard-code your secrets here internalApiKey: "" jwtSecret: "" + smtp: + enabled: false services: dns: cluster.local @@ -118,12 +123,12 @@ services: couchdb: enabled: true replicaCount: 3 - url: "" # only change if pointing to existing couch server - user: "" # only change if pointing to existing couch server - password: "" # only change if pointing to existing couch server + # url: "" # only change if pointing to existing couch server + # user: "" # only change if pointing to existing couch server + # password: "" # only change if pointing to existing couch server port: 5984 storage: 100Mi - + redis: enabled: true # disable if using external redis port: 6379 @@ -131,7 +136,7 @@ services: url: "" # only change if pointing to existing redis cluster and enabled: false password: "budibase" # recommended to override if using built-in redis storage: 100Mi - + objectStore: minio: true browser: true diff --git a/lerna.json b/lerna.json index 37a77037b6..4f08963308 100644 --- a/lerna.json +++ b/lerna.json @@ -1,5 +1,5 @@ { - "version": "0.9.160-alpha.1", + "version": "0.9.167-alpha.8", "npmClient": "yarn", "packages": [ "packages/*" diff --git a/package.json b/package.json index 3596ec7800..eb7b7167f1 100644 --- a/package.json +++ b/package.json @@ -32,6 +32,7 @@ "kill-port": "kill-port 4001", "dev": "yarn run kill-port && lerna link && lerna run --parallel dev:builder --concurrency 1", "dev:noserver": "lerna link && lerna run dev:stack:up && lerna run --parallel dev:builder --concurrency 1 --ignore @budibase/server --ignore @budibase/worker", + "dev:server": "lerna run --parallel dev:builder --concurrency 1 --scope @budibase/worker --scope @budibase/server", "test": "lerna run test", "lint:eslint": "eslint packages", "lint:prettier": "prettier --check \"packages/**/*.{js,svelte}\"", @@ -46,12 +47,17 @@ "build:docker:production": "lerna run build:docker && cd hosting/scripts/linux/ && ./release-to-docker-hub.sh $BUDIBASE_RELEASE_VERSION release && cd -", "build:docker:develop": "node scripts/pinVersions && lerna run build:docker && cd hosting/scripts/linux/ && ./release-to-docker-hub.sh develop && cd -", "release:helm": "./scripts/release_helm_chart.sh", - "multi:enable": "lerna run multi:enable", - "multi:disable": "lerna run multi:disable", - "selfhost:enable": "lerna run selfhost:enable", - "selfhost:disable": "lerna run selfhost:disable", - "localdomain:enable": "lerna run localdomain:enable", - "localdomain:disable": "lerna run localdomain:disable", + "env:multi:enable": "lerna run env:multi:enable", + "env:multi:disable": "lerna run env:multi:disable", + "env:selfhost:enable": "lerna run env:selfhost:enable", + "env:selfhost:disable": "lerna run env:selfhost:disable", + "env:localdomain:enable": "lerna run env:localdomain:enable", + "env:localdomain:disable": "lerna run env:localdomain:disable", + "env:account:enable": "lerna run env:account:enable", + "env:account:disable": "lerna run env:account:disable", + "mode:self": "yarn env:selfhost:enable && yarn env:multi:disable && yarn env:account:disable", + "mode:cloud": "yarn env:selfhost:disable && yarn env:multi:enable && yarn env:account:disable", + "mode:account": "yarn mode:cloud && yarn env:account:enable", "postinstall": "husky install" } } diff --git a/packages/auth/package.json b/packages/auth/package.json index e4282bf6f2..384114b00c 100644 --- a/packages/auth/package.json +++ b/packages/auth/package.json @@ -1,6 +1,6 @@ { "name": "@budibase/auth", - "version": "0.9.160-alpha.1", + "version": "0.9.167-alpha.8", "description": "Authentication middlewares for budibase builder and apps", "main": "src/index.js", "author": "Budibase", diff --git a/packages/auth/src/middleware/authenticated.js b/packages/auth/src/middleware/authenticated.js index 944f3ee9d9..4223e7e395 100644 --- a/packages/auth/src/middleware/authenticated.js +++ b/packages/auth/src/middleware/authenticated.js @@ -42,8 +42,9 @@ module.exports = ( internal = false if (authCookie) { let error = null - const sessionId = authCookie.sessionId, - userId = authCookie.userId + const sessionId = authCookie.sessionId + const userId = authCookie.userId + const session = await getSession(userId, sessionId) if (!session) { error = "No session found" diff --git a/packages/auth/src/security/sessions.js b/packages/auth/src/security/sessions.js index 83ca9d9bcd..93c2d0a9ca 100644 --- a/packages/auth/src/security/sessions.js +++ b/packages/auth/src/security/sessions.js @@ -24,17 +24,24 @@ exports.createASession = async (userId, session) => { await client.store(makeSessionID(userId, sessionId), session, EXPIRY_SECONDS) } -exports.invalidateSessions = async (userId, sessionId = null) => { +exports.invalidateSessions = async (userId, sessionIds = null) => { let sessions = [] - if (sessionId) { - sessions.push({ key: makeSessionID(userId, sessionId) }) - } else { + + // If no sessionIds, get all the sessions for the user + if (!sessionIds) { sessions = await getSessionsForUser(userId) sessions.forEach( session => (session.key = makeSessionID(session.userId, session.sessionId)) ) + } else { + // use the passed array of sessionIds + sessions = Array.isArray(sessionIds) ? sessionIds : [sessionIds] + sessions = sessions.map(sessionId => ({ + key: makeSessionID(userId, sessionId), + })) } + const client = await redis.getSessionClient() const promises = [] for (let session of sessions) { diff --git a/packages/auth/src/utils.js b/packages/auth/src/utils.js index f509a626c1..823fd06322 100644 --- a/packages/auth/src/utils.js +++ b/packages/auth/src/utils.js @@ -7,7 +7,7 @@ const { const jwt = require("jsonwebtoken") const { options } = require("./middleware/passport/jwt") const { createUserEmailView } = require("./db/views") -const { Headers, UserStatus } = require("./constants") +const { Headers, UserStatus, Cookies } = require("./constants") const { getGlobalDB, updateTenantId, @@ -19,6 +19,7 @@ const accounts = require("./cloud/accounts") const { hash } = require("./hashing") const userCache = require("./cache/user") const env = require("./environment") +const { getUserSessions, invalidateSessions } = require("./security/sessions") const APP_PREFIX = DocumentTypes.APP + SEPARATOR @@ -235,3 +236,28 @@ exports.saveUser = async ( } } } + +/** + * Logs a user out from budibase. Re-used across account portal and builder. + */ +exports.platformLogout = async ({ ctx, userId, keepActiveSession }) => { + if (!ctx) throw new Error("Koa context must be supplied to logout.") + + const currentSession = this.getCookie(ctx, Cookies.Auth) + let sessions = await getUserSessions(userId) + + if (keepActiveSession) { + sessions = sessions.filter( + session => session.sessionId !== currentSession.sessionId + ) + } else { + // clear cookies + this.clearCookie(ctx, Cookies.Auth) + this.clearCookie(ctx, Cookies.CurrentApp) + } + + await invalidateSessions( + userId, + sessions.map(({ sessionId }) => sessionId) + ) +} diff --git a/packages/bbui/package.json b/packages/bbui/package.json index cbe7b11e11..777ec44b14 100644 --- a/packages/bbui/package.json +++ b/packages/bbui/package.json @@ -1,7 +1,7 @@ { "name": "@budibase/bbui", "description": "A UI solution used in the different Budibase projects.", - "version": "0.9.160-alpha.1", + "version": "0.9.167-alpha.8", "license": "AGPL-3.0", "svelte": "src/index.js", "module": "dist/bbui.es.js", diff --git a/packages/bbui/src/Table/CellRenderer.svelte b/packages/bbui/src/Table/CellRenderer.svelte index d6a2f3196d..1c3822e51e 100644 --- a/packages/bbui/src/Table/CellRenderer.svelte +++ b/packages/bbui/src/Table/CellRenderer.svelte @@ -5,11 +5,14 @@ import RelationshipRenderer from "./RelationshipRenderer.svelte" import AttachmentRenderer from "./AttachmentRenderer.svelte" import ArrayRenderer from "./ArrayRenderer.svelte" + import InternalRenderer from "./InternalRenderer.svelte" + export let row export let schema export let value export let customRenderers = [] + let renderer const typeMap = { boolean: BooleanRenderer, datetime: DateTimeRenderer, @@ -20,7 +23,9 @@ number: StringRenderer, longform: StringRenderer, array: ArrayRenderer, + internal: InternalRenderer, } + $: type = schema?.type ?? "string" $: customRenderer = customRenderers?.find(x => x.column === schema?.name) $: renderer = customRenderer?.component ?? typeMap[type] ?? StringRenderer diff --git a/packages/bbui/src/Table/InternalRenderer.svelte b/packages/bbui/src/Table/InternalRenderer.svelte new file mode 100644 index 0000000000..0f894ac853 --- /dev/null +++ b/packages/bbui/src/Table/InternalRenderer.svelte @@ -0,0 +1,28 @@ + + +
+ +
+ + diff --git a/packages/bbui/src/Tabs/Tab.svelte b/packages/bbui/src/Tabs/Tab.svelte index 3644280edf..86f2c0ee52 100644 --- a/packages/bbui/src/Tabs/Tab.svelte +++ b/packages/bbui/src/Tabs/Tab.svelte @@ -8,11 +8,19 @@ const selected = getContext("tab") let tab let tabInfo + const setTabInfo = () => { - tabInfo = tab.getBoundingClientRect() - if ($selected.title === title) { - $selected.info = tabInfo - } + // If the tabs are being rendered inside a component which uses + // a svelte transition to enter, then this initial getBoundingClientRect + // will return an incorrect position. + // We just need to get this off the main thread to fix this, by using + // a 0ms timeout. + setTimeout(() => { + tabInfo = tab.getBoundingClientRect() + if ($selected.title === title) { + $selected.info = tabInfo + } + }, 0) } onMount(() => { diff --git a/packages/builder/cypress/integration/createTable.spec.js b/packages/builder/cypress/integration/createTable.spec.js index ccb9ae7929..96a1bd75aa 100644 --- a/packages/builder/cypress/integration/createTable.spec.js +++ b/packages/builder/cypress/integration/createTable.spec.js @@ -31,15 +31,16 @@ context("Create a Table", () => { cy.contains("nameupdated ").should("contain", "nameupdated") }) + it("edits a row", () => { cy.contains("button", "Edit").click({ force: true }) cy.wait(1000) cy.get(".spectrum-Modal input").clear() - cy.get(".spectrum-Modal input").type("RoverUpdated") + cy.get(".spectrum-Modal input").type("Updated") cy.contains("Save").click() cy.contains("Updated").should("have.text", "Updated") }) - + it("deletes a row", () => { cy.get(".spectrum-Checkbox-input").check({ force: true }) cy.contains("Delete 1 row(s)").click() diff --git a/packages/builder/cypress/support/commands.js b/packages/builder/cypress/support/commands.js index c46f6bc9f8..2f859eb478 100644 --- a/packages/builder/cypress/support/commands.js +++ b/packages/builder/cypress/support/commands.js @@ -36,18 +36,11 @@ Cypress.Commands.add("createApp", name => { cy.visit(`localhost:${Cypress.env("PORT")}/builder`) cy.wait(500) cy.contains(/Start from scratch/).click() - cy.get(".spectrum-Modal") - .within(() => { - cy.get("input").eq(0).type(name).should("have.value", name).blur() - cy.get(".spectrum-ButtonGroup").contains("Create app").click() - cy.wait(7000) - }) - .then(() => { - // Because we show the datasource modal on entry, we need to create a table to get rid of the modal in the future - cy.createInitialDatasource("initialTable") - cy.expandBudibaseConnection() - cy.get(".nav-item.selected > .content").should("be.visible") - }) + cy.get(".spectrum-Modal").within(() => { + cy.get("input").eq(0).type(name).should("have.value", name).blur() + cy.get(".spectrum-ButtonGroup").contains("Create app").click() + cy.wait(7000) + }) }) Cypress.Commands.add("deleteApp", () => { @@ -77,22 +70,6 @@ Cypress.Commands.add("createTestTableWithData", () => { cy.addColumn("dog", "age", "Number") }) -Cypress.Commands.add("createInitialDatasource", tableName => { - // Enter table name - cy.get(".spectrum-Modal").within(() => { - cy.contains("Budibase DB").trigger("mouseover").click().click() - cy.wait(1000) - cy.contains("Continue").click() - }) - - cy.get(".spectrum-Modal").within(() => { - cy.wait(1000) - cy.get("input").first().type(tableName).blur() - cy.get(".spectrum-ButtonGroup").contains("Create").click() - }) - cy.contains(tableName).should("be.visible") -}) - Cypress.Commands.add("createTable", tableName => { cy.contains("Budibase DB").click() cy.contains("Create new table").click() diff --git a/packages/builder/package.json b/packages/builder/package.json index b82a9d9554..885b3d0170 100644 --- a/packages/builder/package.json +++ b/packages/builder/package.json @@ -1,6 +1,6 @@ { "name": "@budibase/builder", - "version": "0.9.160-alpha.1", + "version": "0.9.167-alpha.8", "license": "AGPL-3.0", "private": true, "scripts": { @@ -65,10 +65,10 @@ } }, "dependencies": { - "@budibase/bbui": "^0.9.160-alpha.1", - "@budibase/client": "^0.9.160-alpha.1", + "@budibase/bbui": "^0.9.167-alpha.8", + "@budibase/client": "^0.9.167-alpha.8", "@budibase/colorpicker": "1.1.2", - "@budibase/string-templates": "^0.9.160-alpha.1", + "@budibase/string-templates": "^0.9.167-alpha.8", "@sentry/browser": "5.19.1", "@spectrum-css/page": "^3.0.1", "@spectrum-css/vars": "^3.0.1", diff --git a/packages/builder/src/builderStore/dataBinding.js b/packages/builder/src/builderStore/dataBinding.js index dc1e40e517..3389e20d40 100644 --- a/packages/builder/src/builderStore/dataBinding.js +++ b/packages/builder/src/builderStore/dataBinding.js @@ -7,11 +7,17 @@ import { } from "./storeUtils" import { store } from "builderStore" import { queries as queriesStores, tables as tablesStore } from "stores/backend" -import { makePropSafe } from "@budibase/string-templates" +import { + makePropSafe, + isJSBinding, + decodeJSBinding, + encodeJSBinding, +} from "@budibase/string-templates" import { TableNames } from "../constants" // Regex to match all instances of template strings const CAPTURE_VAR_INSIDE_TEMPLATE = /{{([^}]+)}}/g +const CAPTURE_VAR_INSIDE_JS = /\$\("([^")]+)"\)/g const CAPTURE_HBS_TEMPLATE = /{{[\S\s]*?}}/g /** @@ -430,6 +436,15 @@ function replaceBetween(string, start, end, replacement) { * utility function for the readableToRuntimeBinding and runtimeToReadableBinding. */ function bindingReplacement(bindableProperties, textWithBindings, convertTo) { + // Decide from base64 if using JS + const isJS = isJSBinding(textWithBindings) + if (isJS) { + textWithBindings = decodeJSBinding(textWithBindings) + } + + // Determine correct regex to find bindings to replace + const regex = isJS ? CAPTURE_VAR_INSIDE_JS : CAPTURE_VAR_INSIDE_TEMPLATE + const convertFrom = convertTo === "runtimeBinding" ? "readableBinding" : "runtimeBinding" if (typeof textWithBindings !== "string") { @@ -441,7 +456,7 @@ function bindingReplacement(bindableProperties, textWithBindings, convertTo) { .sort((a, b) => { return b.length - a.length }) - const boundValues = textWithBindings.match(CAPTURE_VAR_INSIDE_TEMPLATE) || [] + const boundValues = textWithBindings.match(regex) || [] let result = textWithBindings for (let boundValue of boundValues) { let newBoundValue = boundValue @@ -449,7 +464,7 @@ function bindingReplacement(bindableProperties, textWithBindings, convertTo) { // in the search, working from longest to shortest so always use best match first let searchString = newBoundValue for (let from of convertFromProps) { - if (shouldReplaceBinding(newBoundValue, from, convertTo)) { + if (isJS || shouldReplaceBinding(newBoundValue, from, convertTo)) { const binding = bindableProperties.find(el => el[convertFrom] === from) let idx do { @@ -472,6 +487,12 @@ function bindingReplacement(bindableProperties, textWithBindings, convertTo) { } result = result.replace(boundValue, newBoundValue) } + + // Re-encode to base64 if using JS + if (isJS) { + result = encodeJSBinding(result) + } + return result } diff --git a/packages/builder/src/builderStore/store/automation/Automation.js b/packages/builder/src/builderStore/store/automation/Automation.js index 49928c69a9..af0c03cb5a 100644 --- a/packages/builder/src/builderStore/store/automation/Automation.js +++ b/packages/builder/src/builderStore/store/automation/Automation.js @@ -14,7 +14,7 @@ export default class Automation { } addTestData(data) { - this.automation.testData = data + this.automation.testData = { ...this.automation.testData, ...data } } addBlock(block, idx) { diff --git a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte index e69f5ec204..74e82a2da7 100644 --- a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte +++ b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte @@ -103,7 +103,7 @@ {block?.name?.toUpperCase() || ""} - {#if testResult} + {#if testResult && testResult[0]} resultsModal.show()}> testData[required] @@ -41,7 +45,6 @@ showConfirmButton={true} disabled={isError} onConfirm={() => { - automationStore.actions.addTestDataToAutomation(testData) automationStore.actions.test( $automationStore.selectedAutomation?.automation, testData @@ -53,7 +56,7 @@ >
onChange(e, key)} {bindings} + allowJS={false} /> {/if} {:else if value.customType === "query"} @@ -205,7 +222,10 @@ {bindings} /> {:else if value.customType === "webhookUrl"} - + onChange(e, key)} + value={inputData[key]} + /> {:else if value.customType === "triggerSchema"} onChange(e, key)} value={inputData[key]} /> {:else if value.customType === "code"} @@ -240,6 +260,7 @@ value={inputData[key]} on:change={e => onChange(e, key)} {bindings} + allowJS={false} />
{/if} @@ -247,6 +268,10 @@ {/each} + + + + {#if stepId === "WEBHOOK"} {/if} diff --git a/packages/builder/src/components/automation/SetupPanel/QueryParamSelector.svelte b/packages/builder/src/components/automation/SetupPanel/QueryParamSelector.svelte index 91dc369d80..7a4bda3047 100644 --- a/packages/builder/src/components/automation/SetupPanel/QueryParamSelector.svelte +++ b/packages/builder/src/components/automation/SetupPanel/QueryParamSelector.svelte @@ -39,6 +39,7 @@ type="string" {bindings} fillWidth={true} + allowJS={false} /> {/each} diff --git a/packages/builder/src/components/automation/SetupPanel/RowSelector.svelte b/packages/builder/src/components/automation/SetupPanel/RowSelector.svelte index 1d54c86b4a..fc812e0289 100644 --- a/packages/builder/src/components/automation/SetupPanel/RowSelector.svelte +++ b/packages/builder/src/components/automation/SetupPanel/RowSelector.svelte @@ -1,20 +1,39 @@ @@ -29,6 +37,15 @@ + {:else if schema[configKey].type === "longForm"} +
+ +