diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 7abfe537e9..d1e373003a 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -79,6 +79,8 @@ Component libraries are collections of components as well as the definition of t ### Getting Started For Contributors #### 1. Prerequisites +NodeJS Version `14.x.x` + *yarn -* `npm install -g yarn` *jest* - `npm install -g jest` @@ -177,36 +179,7 @@ To enable this mode, use: yarn mode:account ``` ### CI - -#### PR Job - -After your pr is submitted a github action (can be found at `.github/workflows/budibase_ci.yml`) will run to perform some checks against the changes such as linting, build and test. - -The job will run when changes are pushed to or targetted at `master` and `develop` -#### Release Develop - -To test changes before a release, a prerelease action (can be found at `.github/workflows/release-develop.yml`) will run to build and release develop versions of npm packages and docker images. On each subsequent commit to develop a new alpha version of npm packages will be created and released. - -For example: - -- `feature1` -> `develop` = `v0.9.160-alpha.1` -- `feature2` -> `develop` = `v0.9.160-alpha.0` - -The job will run when changes are pushed to `develop` -#### Release Job - -To release changes a release job (can be found at `.github/workflows/release.yml`) will run to create final versions of npm packages and docker images. - -Following the example above: - -- `develop` -> `master` = `v0.9.160` - -The job will run when changes are pushed to `master` - -#### Release Self Host Job - -To release the self hosted version of docker images, an additional job (can be found at `.github/workflows/release-selfhost.yml`) must be ran manually. This will releaae docker images to docker hub under the tag `latest` to be picked up by self hosted installations. - + An overview of the CI pipelines can be found [here](./workflows/README.md) ### Troubleshooting Sometimes, things go wrong. This can be due to incompatible updates on the budibase platform. To clear down your development environment and start again follow **Step 6. Cleanup**, then proceed from **Step 3. Install and Build** in the setup guide above. You should have a fresh Budibase installation. diff --git a/.github/stale.yml b/.github/stale.yml index 3112dfa8e7..5875ed1282 100644 --- a/.github/stale.yml +++ b/.github/stale.yml @@ -1,7 +1,9 @@ -# Number of days of inactivity before an issue becomes stale +# Configuration for probot-stale - https://github.com/probot/stale +# Number of days of inactivity before an Issue or Pull Request becomes stale daysUntilStale: 60 -# Number of days of inactivity before a stale issue is closed -daysUntilClose: 7 +# Number of days of inactivity before an Issue or Pull Request with the stale label is closed. +# Set to false to disable. If disabled, issues still need to be closed manually, but will remain marked as stale. +daysUntilClose: false # Issues with these labels will never be considered stale exemptLabels: - pinned diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 0000000000..d2fcd16bb0 --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,93 @@ + +# Budibase CI Pipelines + +Welcome to the budibase CI pipelines directory. This document details what each of the CI pipelines are for, and come common combinations. + +## All CI Pipelines + +### Note +- When running workflow dispatch jobs, ensure you always run them off the `master` branch. It defaults to `develop`, so double check before running any jobs. + +### Standard CI Build Job (budibase_ci.yml) +Triggers: +- PR or push to develop +- PR or push to master + +The standard CI Build job is what runs when you raise a PR to develop or master. +- Installs all dependencies, +- builds the project +- run the unit tests +- Generate test coverage metrics with codecov +- Run the cypress tests + +### Release Develop Job (release-develop.yml) +Triggers: +- Push to develop + +The job responsible for building, tagging and pushing docker images out to the test and staging environments. +- Installs all dependencies +- builds the project +- run the unit tests +- publish the budibase JS packages under a prerelease tag to NPM +- build, tag and push docker images under the `develop` tag to docker hub + +These images will then be pulled by the test and staging environments, updating the latest automatically. Discord notifications are sent to the #infra channel when this occurs. + +### Release Job (release.yml) +Triggers: +- Push to master + +This job is responsible for building and pushing the latest code to NPM and docker hub, so that it can be deployed. +- Installs all dependencies +- builds the project +- run the unit tests +- publish the budibase JS packages under a release tag to NPM (always incremented by patch versions) +- build, tag and push docker images under the `v.x.x.x` (the tag of the NPM release) tag to docker hub + +### Release Selfhost Job (release-selfhost.yml) +Triggers: +- Manual Workflow Dispatch Trigger + +This job is responsible for delivering the latest version of budibase to those that are self-hosting. + +This job relies on the release job to have run first, so the latest image is pushed to dockerhub. This job then will pull the latest version from `lerna.json` and try to find an image in dockerhub corresponding to that version. For example, if the version in `lerna.json` is `1.0.0`: +- Pull the images for all budibase services tagged `v1.0.0` from dockerhub +- Tag these images as `latest` +- Push them back to dockerhub. This now means anyone who pulls `latest` (self hosters using docker-compose) will get the latest version. +- Build and release the budibase helm chart for kubernetes users +- Perform a github release with the latest version. You can see previous releases here (https://github.com/Budibase/budibase/releases) + + +### Cloud Deploy (deploy-cloud.yml) +Triggers: +- Manual Workflow Dispatch Trigger + +This job is responsible for deploying to our production, cloud kubernetes environment. You must run the release job first, to ensure that the latest images have been built and pushed to docker hub. You can also manually enter a version number for this job, so you can perform rollbacks or upgrade to a specific version. After kicking off this job, the following will occur: + +- Checks out the master branch +- Pulls the latest `values.yaml` from budibase infra, a private repo containing budibases infrastructure configuration +- Gets the latest budibase version from `lerna.json`, if it hasn't been specified in the workflow when you kicked it off +- Configures AWS Credentials +- Deploys the helm chart in the budibase repo to our production EKS cluster, injecting the `values.yaml` we pulled from budibase-infra +- Fires off a discord webhook in the #infra channel to show that the deployment completely successfully. + +## Common Workflows + +### Deploy Changes to Production (Release) +- Merge `develop` into `master` +- Wait for budibase CI job and release job to run +- Run cloud deploy job +- Run release selfhost job + +### Deploy Changes to Production (Hotfix) +- Branch off `master` +- Perform your hotfix +- Merge back into `master` +- Wait for budibase CI job and release job to run +- Run cloud deploy job +- Run release selfhost job + +### Rollback A Bad Cloud Deployment +- Kick off cloud deploy job +- Ensure you are running off master +- Enter the version number of the last known good version of budibase. For example `1.0.0` \ No newline at end of file diff --git a/.github/workflows/budibase_ci.yml b/.github/workflows/budibase_ci.yml index 5c4a111e23..7e95115415 100644 --- a/.github/workflows/budibase_ci.yml +++ b/.github/workflows/budibase_ci.yml @@ -41,4 +41,6 @@ jobs: files: ./packages/server/coverage/clover.xml name: codecov-umbrella verbose: true + + # TODO: parallelise this - run: yarn test:e2e:ci diff --git a/.github/workflows/deploy-cloud.yaml b/.github/workflows/deploy-cloud.yaml index 26422a2e7b..d54e6c9c68 100644 --- a/.github/workflows/deploy-cloud.yaml +++ b/.github/workflows/deploy-cloud.yaml @@ -12,6 +12,12 @@ jobs: runs-on: ubuntu-latest steps: + - name: Fail if branch is not master + if: github.ref != 'refs/heads/master' + run: | + echo "Ref is not master, you must run this job from master." + exit 1 + - uses: actions/checkout@v2 - name: Pull values.yaml from budibase-infra diff --git a/.github/workflows/deploy-preprod.yml b/.github/workflows/deploy-preprod.yml index ce789e71ec..5b3282313c 100644 --- a/.github/workflows/deploy-preprod.yml +++ b/.github/workflows/deploy-preprod.yml @@ -23,16 +23,19 @@ jobs: aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} aws-region: eu-west-1 - - name: 'Get Previous tag' - id: previoustag - uses: "WyriHaximus/github-action-get-previous-tag@v1" + - name: Get the latest budibase release version + id: version + run: | + release_version=$(cat lerna.json | jq -r '.version') + echo "RELEASE_VERSION=$release_version" >> $GITHUB_ENV - # - name: Pull values.yaml from budibase-infra - # run: | - # curl -H "Authorization: token ${{ secrets.GH_PERSONAL_TOKEN }}" \ [c3a7a9d12] - # -H 'Accept: application/vnd.github.v3.raw' \ - # -o values.preprod.yaml \ - # -L https://api.github.com/repos/budibase/budibase-infra/contents/kubernetes/values.preprod.yaml + - name: Pull values.yaml from budibase-infra + run: | + curl -H "Authorization: token ${{ secrets.GH_PERSONAL_TOKEN }}" \ + -H 'Accept: application/vnd.github.v3.raw' \ + -o values.preprod.yaml \ + -L https://api.github.com/repos/budibase/budibase-infra/contents/kubernetes/budibase-preprod/values.yaml + wc -l values.preprod.yaml - name: Deploy to Preprod Environment uses: deliverybot/helm@v1 @@ -41,13 +44,17 @@ jobs: namespace: budibase chart: charts/budibase token: ${{ github.token }} + helm: helm3 values: | globals: - appVersion: ${{ steps.previoustag.outputs.tag }} - # value-files: >- - # [ - # "charts/budibase/values.yaml" - # ] + appVersion: v${{ env.RELEASE_VERSION }} + ingress: + enabled: true + nginx: true + value-files: >- + [ + "values.preprod.yaml" + ] env: KUBECONFIG_FILE: '${{ secrets.PREPROD_KUBECONFIG }}' diff --git a/.github/workflows/release-develop.yml b/.github/workflows/release-develop.yml index b0f052f68f..4467cd6c81 100644 --- a/.github/workflows/release-develop.yml +++ b/.github/workflows/release-develop.yml @@ -4,6 +4,16 @@ on: push: branches: - develop + paths: + - '.aws/**' + - '.github/**' + - 'charts/**' + - 'packages/**' + - 'scripts/**' + - 'package.json' + - 'yarn.lock' + - 'package.json' + - 'yarn.lock' env: POSTHOG_TOKEN: ${{ secrets.POSTHOG_TOKEN }} diff --git a/.github/workflows/release-selfhost.yml b/.github/workflows/release-selfhost.yml index 3e2e5ca6b7..e842da9d7c 100644 --- a/.github/workflows/release-selfhost.yml +++ b/.github/workflows/release-selfhost.yml @@ -9,7 +9,8 @@ jobs: steps: - uses: actions/checkout@v2 - with: + with: + node-version: 14.x fetch_depth: 0 - name: Tag and release Docker images (Self Host) @@ -37,33 +38,39 @@ jobs: DOCKER_PASSWORD: ${{ secrets.DOCKER_API_KEY }} SELFHOST_TAG: latest + - name: Build CLI executables + run: | + pushd packages/cli + yarn + yarn build + popd + - name: Setup Helm uses: azure/setup-helm@v1 id: helm-install - # - name: Build and release helm chart - # run: | - # git config user.name "Budibase Helm Bot" - # git config user.email "<>" - # mv budibase-${{ env.RELEASE_VERSION }}.tgz docs - # helm repo index docs - # git checkout gh-pages - # git add -A - # git commit -m "Helm Release: ${{ env.RELEASE_VERSION }}" - # git push - # env: - # GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - # - name: Deploy - # uses: peaceiris/actions-gh-pages@v3 - # with: - # github_token: ${{ secrets.GITHUB_TOKEN }} - # publish_dir: ./public - # full_commit_message: "Helm Release: ${{ env.RELEASE_VERSION }}" + - name: Build and release helm chart + run: | + git config user.name "Budibase Helm Bot" + git config user.email "<>" + git pull + helm package charts/budibase + git checkout gh-pages + mv *.tgz docs + helm repo index docs + git add -A + git commit -m "Helm Release: ${{ env.RELEASE_VERSION }}" + git push + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - name: Perform Github Release uses: softprops/action-gh-release@v1 with: name: v${{ env.RELEASE_VERSION }} tag_name: v${{ env.RELEASE_VERSION }} - generate_release_notes: true \ No newline at end of file + generate_release_notes: true + files: | + packages/cli/build/cli-win.exe + packages/cli/build/cli-linux + packages/cli/build/cli-macos \ No newline at end of file diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 859d89f18b..82848c78e4 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -3,7 +3,17 @@ name: Budibase Release on: push: branches: - - test + - master + paths: + - '.aws/**' + - '.github/**' + - 'charts/**' + - 'packages/**' + - 'scripts/**' + - 'package.json' + - 'yarn.lock' + - 'package.json' + - 'yarn.lock' env: POSTHOG_TOKEN: ${{ secrets.POSTHOG_TOKEN }} @@ -56,34 +66,3 @@ jobs: DOCKER_USER: ${{ secrets.DOCKER_USERNAME }} DOCKER_PASSWORD: ${{ secrets.DOCKER_API_KEY }} BUDIBASE_RELEASE_VERSION: ${{ steps.previoustag.outputs.tag }} - - # - name: Pull values.yaml from budibase-infra - # run: | - # curl -H "Authorization: token ${{ secrets.GH_PERSONAL_TOKEN }}" \ [c3a7a9d12] - # -H 'Accept: application/vnd.github.v3.raw' \ - # -o values.preprod.yaml \ - # -L https://api.github.com/repos/budibase/budibase-infra/contents/kubernetes/values.preprod.yaml - - # - name: Deploy to Preprod Environment - # uses: deliverybot/helm@v1 - # with: - # release: budibase-preprod - # namespace: budibase - # chart: charts/budibase - # token: ${{ github.token }} - # values: | - # globals: - # appVersion: ${{ steps.previoustag.outputs.tag }} - # # value-files: >- - # # [ - # # "charts/budibase/values.yaml" - # # ] - # env: - # KUBECONFIG_FILE: '${{ secrets.PREPROD_KUBECONFIG }}' - - # - name: Discord Webhook Action - # uses: tsickert/discord-webhook@v4.0.0 - # with: - # webhook-url: ${{ secrets.PROD_DEPLOY_WEBHOOK_URL }} - # content: "Preprod Deployment Complete: ${{ env.RELEASE_VERSION }} deployed to Budibase Pre-prod." - # embed-title: ${{ env.RELEASE_VERSION }} diff --git a/.github/workflows/smoke_test.yaml b/.github/workflows/smoke_test.yaml new file mode 100644 index 0000000000..745fed1306 --- /dev/null +++ b/.github/workflows/smoke_test.yaml @@ -0,0 +1,46 @@ +name: Budibase Smoke Test + +on: + workflow_dispatch: + +jobs: + release: + runs-on: ubuntu-latest + + steps: + - uses: actions/checkout@v2 + - name: Use Node.js 14.x + uses: actions/setup-node@v1 + with: + node-version: 14.x + - run: yarn + - run: yarn bootstrap + - run: yarn build + - name: Pull cypress.env.yaml from budibase-infra + run: | + curl -H "Authorization: token ${{ secrets.GH_PERSONAL_TOKEN }}" \ + -H 'Accept: application/vnd.github.v3.raw' \ + -o packages/builder/cypress.env.json \ + -L https://api.github.com/repos/budibase/budibase-infra/contents/test/cypress.env.json + wc -l packages/builder/cypress.env.json + - run: yarn test:e2e:ci + env: + CI: true + name: Budibase CI + + # TODO: upload recordings to s3 + # - name: Configure AWS Credentials + # uses: aws-actions/configure-aws-credentials@v1 + # with: + # aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + # aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + # aws-region: eu-west-1 + + # TODO look at cypress reporters + # - name: Discord Webhook Action + # uses: tsickert/discord-webhook@v4.0.0 + # with: + # webhook-url: ${{ secrets.PROD_DEPLOY_WEBHOOK_URL }} + # content: "Production Deployment Complete: ${{ env.RELEASE_VERSION }} deployed to Budibase Cloud." + # embed-title: ${{ env.RELEASE_VERSION }} + diff --git a/.gitignore b/.gitignore index 6652344b0b..2376553d6d 100644 --- a/.gitignore +++ b/.gitignore @@ -93,4 +93,5 @@ hosting/.generated-envoy.dev.yaml # Sublime text *.sublime-project *.sublime-workspace -packages/builder/cypress.env.json +packages/builder/cypress.env.json +bin/ diff --git a/charts/budibase/Chart.lock b/charts/budibase/Chart.lock new file mode 100644 index 0000000000..75b9de07b5 --- /dev/null +++ b/charts/budibase/Chart.lock @@ -0,0 +1,9 @@ +dependencies: +- name: couchdb + repository: https://apache.github.io/couchdb-helm + version: 3.3.4 +- name: ingress-nginx + repository: https://kubernetes.github.io/ingress-nginx + version: 4.0.13 +digest: sha256:20892705c2d8e64c98257d181063a514ac55013e2b43399a6e54868a97f97845 +generated: "2021-12-30T18:55:30.878411Z" diff --git a/charts/budibase/Chart.yaml b/charts/budibase/Chart.yaml index cec5af2914..8c9d44f201 100644 --- a/charts/budibase/Chart.yaml +++ b/charts/budibase/Chart.yaml @@ -11,14 +11,14 @@ sources: - https://github.com/Budibase/budibase - https://budibase.com type: application -version: 1.0.0 -appVersion: 1.0.20 +version: 0.2.5 +appVersion: 1.0.25 dependencies: - name: couchdb version: 3.3.4 repository: https://apache.github.io/couchdb-helm condition: services.couchdb.enabled - name: ingress-nginx - version: 3.35.0 - repository: https://github.com/kubernetes/ingress-nginx + version: 4.0.13 + repository: https://kubernetes.github.io/ingress-nginx condition: ingress.nginx diff --git a/charts/budibase/charts/ingress-nginx-3.35.0.tgz b/charts/budibase/charts/ingress-nginx-3.35.0.tgz deleted file mode 100644 index ee5214c497..0000000000 Binary files a/charts/budibase/charts/ingress-nginx-3.35.0.tgz and /dev/null differ diff --git a/charts/budibase/charts/ingress-nginx-4.0.13.tgz b/charts/budibase/charts/ingress-nginx-4.0.13.tgz new file mode 100644 index 0000000000..1e34215c5f Binary files /dev/null and b/charts/budibase/charts/ingress-nginx-4.0.13.tgz differ diff --git a/charts/budibase/templates/proxy-service-service.yaml b/charts/budibase/templates/proxy-service-service.yaml index 8f14d97862..bf2b199ee5 100644 --- a/charts/budibase/templates/proxy-service-service.yaml +++ b/charts/budibase/templates/proxy-service-service.yaml @@ -9,12 +9,11 @@ metadata: app.kubernetes.io/name: budibase-proxy name: proxy-service spec: - type: NodePort ports: - - port: {{ .Values.services.proxy.port }} + - name: {{ .Values.services.proxy.port | quote }} + port: {{ .Values.services.proxy.port }} targetPort: {{ .Values.services.proxy.port }} - protocol: TCP selector: app.kubernetes.io/name: budibase-proxy status: - loadBalancer: {} + loadBalancer: {} \ No newline at end of file diff --git a/charts/budibase/values.yaml b/charts/budibase/values.yaml index b634f1581f..4666d01c70 100644 --- a/charts/budibase/values.yaml +++ b/charts/budibase/values.yaml @@ -40,7 +40,7 @@ service: port: 10000 ingress: - enabled: false + enabled: true aws: false nginx: true certificateArn: "" @@ -302,4 +302,4 @@ couchdb: initialDelaySeconds: 0 periodSeconds: 10 successThreshold: 1 - timeoutSeconds: 1 \ No newline at end of file + timeoutSeconds: 1 diff --git a/i18n/README.fr.md b/i18n/README.fr.md new file mode 100644 index 0000000000..12abd4d073 --- /dev/null +++ b/i18n/README.fr.md @@ -0,0 +1,214 @@ +

+ + Budibase + +

+

+ Budibase +

+ +

+ La plateform low-code que vous aimerez utiliser +

+

+ Budibase est une plateforme low-code open source et c'est la façon la plus facile de créer des outils internes qui améliore la productivité. +

+ +

+ 🤖 🎨 🚀 +

+
+ +

+ Budibase design ui +

+ +

+ + GitHub toutes les releases + + + GitHub release (par ordre chronologique) + + + Suivre @budibase + + Code de conduite + + + +

+ +

+ Commencer + · + Documentation + · + Demandes d'amélioration + · + Signaler un bug + · + Support: Discussions +

+ +

+## ✨ Fontionnalités + +### Construire et déployer un vrai logiciel +Contrairement à d'autres plateformes, avec Budibase vous construisez et déployez des applications one-page. Les applications Budibase sont très perfomantes et peuvent être designées de manière responsive, offrant ainsi à vos utilisateurs une expérience exceptionnelle. +

+ +### Source libre et extensible +Budibase est un logiciel libre - sous licence GPL v3. Cela devrait vous rassurer sur le fait que Budibase sera toujours là. Vous pouvez également coder dans Budibase ou le forker et apporter des modifications à votre guise, ce qui en fera une expérience conviviale pour les développeurs. +

+ +### Importer les données ou partir de zéro +Budibase peut tirer ses données de plusieurs sources, dont MongoDB, CouchDB, PostgreSQL, MySQL, Airtable, S3, DynamoDB ou une API REST. Et contrairement à d'autres plateformes, avec Budibase, vous pouvez partir de zéro et créer des applications métier sans aucune source de données. [Demander une nouvelle source de données](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas). + +

+ Budibase data +

+

+ +### Concevoir et créer des applications à l'aide de composants prédéfinis. + +Budibase est livré avec des composants joliment conçus et puissants que vous pouvez utiliser comme des blocs de construction pour bâtir votre interface utilisateur. Nous exposons également un grand nombre de vos options de style CSS préférées afin que vous puissiez faire preuve d'une créativité accrue. [Demander un nouveau composant](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas). + +

+ Budibase design +

+

+ +### Automatiser les processus, intégrer d'autres outils et se connecter à des webhooks +Gagnez du temps en automatisant les processus manuels et les flux de travail. Qu'il s'agisse de se connecter à des webhooks ou d'automatiser des e-mails, il suffit de dire à Budibase ce qu'il doit faire et de le laisser travailler pour vous. Vous pouvez aisément [créer une nouvelle automatisation pour Budibase ici](https://github.com/Budibase/automations) ou [Demander une nouvelle automatisation](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas). + +

+ Budibase automations +

+

+ +### Intégration avec vos outils préférés +Budibase s'intègre à un certain nombre d'outils populaires, ce qui vous permet de créer des applications qui s'adaptent parfaitement à votre pile technologique. + +

+ Budibase integrations +

+

+ +### Paradis des admins +Budibase est conçu pour évoluer. Avec Budibase, vous pouvez vous auto-héberger sur votre propre infrastructure et gérer globalement les utilisateurs, l'accueil, le SMTP, les applications, les groupes, l'apparence et plus encore. Vous pouvez également fournir aux utilisateurs/groupes un portail d'applications et confier la gestion des utilisateurs au responsable du groupe. + +- Regardez la vidéo de promotion: https://youtu.be/xoljVpty_Kw + +


+ +## 🏁 Commencer + + + +Déployez Budibase en auto-hébergement dans votre infrastructure existante, en utilisant Docker, Kubernetes et Digital Ocean. +Ou utilisez Budibase Cloud si vous n'avez pas besoin de vous auto-héberger, et que vous souhaitez démarrer rapidement. + +### [Commencer avec Budibase](https://budibase.com) + + +

+ +## 🎓 Apprendre Budibase + +La documentation Budibase [est ic](https://docs.budibase.com). +
+ + +

+ +## 💬 Communauté + +Si vous avez une question ou si vous souhaitez discuter avec d'autres utilisateurs de Budibase et rejoindre notre communauté, veuillez vous rendre à l'adresse suivante : [Discussions Github](https://github.com/Budibase/budibase/discussions) + +


+ + +## ❗ Code de conduite + +Budibase s'engage à offrir à chacun une expérience accueillante, diversifiée et exempte de harcèlement. Nous attendons de tous les membres de la communauté Budibase qu'ils se conforment aux principes de notre [**Code de conduite**](https://github.com/Budibase/budibase/blob/HEAD/.github/CODE_OF_CONDUCT.md). Merci de le lire. +
+ + +

+ + +## 🙌 Contribuer à Budibase + +Qu'il s'agisse d'ouvrir un rapport de bug ou de créer une Pull request, toute contribution est appréciée et bienvenue. Si vous envisagez de mettre en œuvre une nouvelle fonctionnalité ou de modifier l'API, veuillez d'abord créer un Issue. Nous pourrons ainsi nous assurer que votre travail n'est pas vain. + +### Vous ne savez pas par où commencer ? +Un bon endroit pour commencer à contribuer, c'est ici : [Projets en cours](https://github.com/Budibase/budibase/projects/22). + +### Comment le repo est-il organisé ? +Budibase est une monorepo gérée par lerna. Lerna gère la construction et la publication des paquets de Budibase. Voici, à un haut niveau, les paquets qui composent Budibase. + +- [packages/builder](https://github.com/Budibase/budibase/tree/HEAD/packages/builder) - contient le code pour l'application svelte côté client du budibase builder. + +- [packages/client](https://github.com/Budibase/budibase/tree/HEAD/packages/client) - Un module qui s'exécute dans le navigateur et qui est chargé de lire les définitions JSON et de créer des applications web vivantes à partir de celles-ci.. + +- [packages/server](https://github.com/Budibase/budibase/tree/HEAD/packages/server) - Le serveur budibase. Cette application Koa est responsable de servir le JS pour les applications builder et budibase, ainsi que de fournir l'API pour l'interaction avec la base de données et le système de fichiers. + +Pour plus d'informations, voir [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md) + +

+ + +## 📝 Licence + +Budibase est open source, sous licence de [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html). Les bibliothèques du client et des composants sont sous licence [MPL](https://directory.fsf.org/wiki/License:MPL-2.0) - afin que les applications que vous créez puissent être utilisées sous licence comme vous le souhaitez. + +

+ +## ⭐ Stargazers dans le temps + +[![Stargazers dans le temps](https://starchart.cc/Budibase/budibase.svg)](https://starchart.cc/Budibase/budibase) + +Si vous rencontrez des problèmes entre les mises à jour du builder, veuillez utiliser le guide suivant [ici](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md#troubleshooting) pour nettoyer votre environnement. + +

+ +## Contributeurs ✨ + +Merci à ces personnes merveilleuses ([emoji key](https://allcontributors.org/docs/en/emoji-key)): + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Martin McKeaveney
💻 📖 ⚠️ 🚇
Michael Drury
📖 💻 ⚠️ 🚇
Andrew Kingston
📖 💻 ⚠️ 🎨
Michael Shanks
📖 💻 ⚠️
Kevin Åberg Kultalahti
📖 💻 ⚠️
Joe
📖 💻 🖋 🎨
Rory Powell
💻 📖 ⚠️

Peter Clement
💻 📖 ⚠️
Conor_Mack
💻 ⚠️
pngwn
💻 ⚠️
HugoLd
💻
victoriasloan
💻
yashank09
💻
SOVLOOKUP
💻
seoulaja
🌍
Maurits Lourens
⚠️ 💻

Rory Powell
🚇 ⚠️ 💻
+ + + + + + +Ce projet suit la spécification [all-contributors](https://allcontributors.org/docs/fr/overview). Les contributions de toute nature sont les bienvenues ! diff --git a/lerna.json b/lerna.json index 6bc599b3e9..4240dd788a 100644 --- a/lerna.json +++ b/lerna.json @@ -1,5 +1,5 @@ { - "version": "1.0.27-alpha.0", + "version": "1.0.44-alpha.1", "npmClient": "yarn", "packages": [ "packages/*" diff --git a/packages/auth/.gitignore b/packages/backend-core/.gitignore similarity index 100% rename from packages/auth/.gitignore rename to packages/backend-core/.gitignore diff --git a/packages/auth/LICENSE b/packages/backend-core/LICENSE similarity index 100% rename from packages/auth/LICENSE rename to packages/backend-core/LICENSE diff --git a/packages/auth/README.md b/packages/backend-core/README.md similarity index 78% rename from packages/auth/README.md rename to packages/backend-core/README.md index 4c6a474b5b..f36f80f84d 100644 --- a/packages/auth/README.md +++ b/packages/backend-core/README.md @@ -4,7 +4,7 @@ This library contains core functionality, like auth and security features which are shared between backend services. #### Note about top level JS files -For the purposes of being able to do say `require("@budibase/auth/permissions")` we need to +For the purposes of being able to do say `require("@budibase/backend-core/permissions")` we need to specify the exports at the top-level of the module. For these files they should be limited to a single `require` of the file that should diff --git a/packages/auth/accounts.js b/packages/backend-core/accounts.js similarity index 100% rename from packages/auth/accounts.js rename to packages/backend-core/accounts.js diff --git a/packages/backend-core/auth.js b/packages/backend-core/auth.js new file mode 100644 index 0000000000..bbfe3d41dd --- /dev/null +++ b/packages/backend-core/auth.js @@ -0,0 +1 @@ +module.exports = require("./src/auth") diff --git a/packages/auth/cache.js b/packages/backend-core/cache.js similarity index 100% rename from packages/auth/cache.js rename to packages/backend-core/cache.js diff --git a/packages/auth/constants.js b/packages/backend-core/constants.js similarity index 100% rename from packages/auth/constants.js rename to packages/backend-core/constants.js diff --git a/packages/auth/db.js b/packages/backend-core/db.js similarity index 100% rename from packages/auth/db.js rename to packages/backend-core/db.js diff --git a/packages/auth/deprovision.js b/packages/backend-core/deprovision.js similarity index 100% rename from packages/auth/deprovision.js rename to packages/backend-core/deprovision.js diff --git a/packages/backend-core/middleware.js b/packages/backend-core/middleware.js new file mode 100644 index 0000000000..30fec96239 --- /dev/null +++ b/packages/backend-core/middleware.js @@ -0,0 +1 @@ +module.exports = require("./src/middleware") diff --git a/packages/backend-core/objectStore.js b/packages/backend-core/objectStore.js new file mode 100644 index 0000000000..3ee433f224 --- /dev/null +++ b/packages/backend-core/objectStore.js @@ -0,0 +1,4 @@ +module.exports = { + ...require("./src/objectStore"), + ...require("./src/objectStore/utils"), +} diff --git a/packages/auth/package.json b/packages/backend-core/package.json similarity index 87% rename from packages/auth/package.json rename to packages/backend-core/package.json index e50241dd65..f01880d87f 100644 --- a/packages/auth/package.json +++ b/packages/backend-core/package.json @@ -1,7 +1,7 @@ { - "name": "@budibase/auth", - "version": "1.0.27-alpha.0", - "description": "Authentication middlewares for budibase builder and apps", + "name": "@budibase/backend-core", + "version": "1.0.44-alpha.1", + "description": "Budibase backend core libraries used in server and worker", "main": "src/index.js", "author": "Budibase", "license": "GPL-3.0", diff --git a/packages/auth/permissions.js b/packages/backend-core/permissions.js similarity index 100% rename from packages/auth/permissions.js rename to packages/backend-core/permissions.js diff --git a/packages/auth/redis.js b/packages/backend-core/redis.js similarity index 100% rename from packages/auth/redis.js rename to packages/backend-core/redis.js diff --git a/packages/auth/roles.js b/packages/backend-core/roles.js similarity index 100% rename from packages/auth/roles.js rename to packages/backend-core/roles.js diff --git a/packages/auth/scripts/jestSetup.js b/packages/backend-core/scripts/jestSetup.js similarity index 100% rename from packages/auth/scripts/jestSetup.js rename to packages/backend-core/scripts/jestSetup.js diff --git a/packages/auth/sessions.js b/packages/backend-core/sessions.js similarity index 100% rename from packages/auth/sessions.js rename to packages/backend-core/sessions.js diff --git a/packages/auth/src/index.js b/packages/backend-core/src/auth.js similarity index 52% rename from packages/auth/src/index.js rename to packages/backend-core/src/auth.js index 4aa2c8ab96..7f66d887ae 100644 --- a/packages/auth/src/index.js +++ b/packages/backend-core/src/auth.js @@ -1,7 +1,6 @@ const passport = require("koa-passport") const LocalStrategy = require("passport-local").Strategy const JwtStrategy = require("passport-jwt").Strategy -const { StaticDatabases } = require("./db/utils") const { getGlobalDB } = require("./tenancy") const { jwt, @@ -14,8 +13,6 @@ const { appTenancy, authError, } = require("./middleware") -const { setDB } = require("./db") -const userCache = require("./cache/user") // Strategies passport.use(new LocalStrategy(local.options, local.authenticate)) @@ -36,36 +33,13 @@ passport.deserializeUser(async (user, done) => { }) module.exports = { - init(pouch) { - setDB(pouch) - }, - db: require("./db/utils"), - redis: { - Client: require("./redis"), - utils: require("./redis/utils"), - }, - objectStore: { - ...require("./objectStore"), - ...require("./objectStore/utils"), - }, - utils: { - ...require("./utils"), - ...require("./hashing"), - }, - auth: { - buildAuthMiddleware: authenticated, - passport, - google, - oidc, - jwt: require("jsonwebtoken"), - buildTenancyMiddleware: tenancy, - buildAppTenancyMiddleware: appTenancy, - auditLog, - authError, - }, - cache: { - user: userCache, - }, - StaticDatabases, - constants: require("./constants"), + buildAuthMiddleware: authenticated, + passport, + google, + oidc, + jwt: require("jsonwebtoken"), + buildTenancyMiddleware: tenancy, + buildAppTenancyMiddleware: appTenancy, + auditLog, + authError, } diff --git a/packages/auth/src/cache/appMetadata.js b/packages/backend-core/src/cache/appMetadata.js similarity index 100% rename from packages/auth/src/cache/appMetadata.js rename to packages/backend-core/src/cache/appMetadata.js diff --git a/packages/auth/src/cache/user.js b/packages/backend-core/src/cache/user.js similarity index 100% rename from packages/auth/src/cache/user.js rename to packages/backend-core/src/cache/user.js diff --git a/packages/auth/src/cloud/accounts.js b/packages/backend-core/src/cloud/accounts.js similarity index 100% rename from packages/auth/src/cloud/accounts.js rename to packages/backend-core/src/cloud/accounts.js diff --git a/packages/auth/src/cloud/api.js b/packages/backend-core/src/cloud/api.js similarity index 100% rename from packages/auth/src/cloud/api.js rename to packages/backend-core/src/cloud/api.js diff --git a/packages/auth/src/constants.js b/packages/backend-core/src/constants.js similarity index 92% rename from packages/auth/src/constants.js rename to packages/backend-core/src/constants.js index 363274eda5..8e6b01608e 100644 --- a/packages/auth/src/constants.js +++ b/packages/backend-core/src/constants.js @@ -8,6 +8,7 @@ exports.Cookies = { Auth: "budibase:auth", Init: "budibase:init", OIDC_CONFIG: "budibase:oidc:config", + RETURN_URL: "budibase:returnurl", } exports.Headers = { @@ -16,6 +17,7 @@ exports.Headers = { APP_ID: "x-budibase-app-id", TYPE: "x-budibase-type", TENANT_ID: "x-budibase-tenant-id", + TOKEN: "x-budibase-token", } exports.GlobalRoles = { diff --git a/packages/auth/src/db/Replication.js b/packages/backend-core/src/db/Replication.js similarity index 100% rename from packages/auth/src/db/Replication.js rename to packages/backend-core/src/db/Replication.js diff --git a/packages/auth/src/db/constants.js b/packages/backend-core/src/db/constants.js similarity index 100% rename from packages/auth/src/db/constants.js rename to packages/backend-core/src/db/constants.js diff --git a/packages/auth/src/db/index.js b/packages/backend-core/src/db/index.js similarity index 100% rename from packages/auth/src/db/index.js rename to packages/backend-core/src/db/index.js diff --git a/packages/auth/src/db/utils.js b/packages/backend-core/src/db/utils.js similarity index 97% rename from packages/auth/src/db/utils.js rename to packages/backend-core/src/db/utils.js index 2bad09c306..5830de4721 100644 --- a/packages/auth/src/db/utils.js +++ b/packages/backend-core/src/db/utils.js @@ -224,8 +224,15 @@ exports.getAllDbs = async () => { } } let couchUrl = `${exports.getCouchUrl()}/_all_dbs` - if (env.MULTI_TENANCY) { - let tenantId = getTenantId() + let tenantId = getTenantId() + if (!env.MULTI_TENANCY || tenantId == DEFAULT_TENANT_ID) { + // just get all DBs when: + // - single tenancy + // - default tenant + // - apps dbs don't contain tenant id + // - non-default tenant dbs are filtered out application side in getAllApps + await addDbs(couchUrl) + } else { // get prod apps await addDbs( exports.getStartEndKeyURL(couchUrl, DocumentTypes.APP, tenantId) @@ -236,9 +243,6 @@ exports.getAllDbs = async () => { ) // add global db name dbs.push(getGlobalDBName(tenantId)) - } else { - // just get all DBs in self host - await addDbs(couchUrl) } return dbs } diff --git a/packages/auth/src/db/views.js b/packages/backend-core/src/db/views.js similarity index 100% rename from packages/auth/src/db/views.js rename to packages/backend-core/src/db/views.js diff --git a/packages/auth/src/environment.js b/packages/backend-core/src/environment.js similarity index 100% rename from packages/auth/src/environment.js rename to packages/backend-core/src/environment.js diff --git a/packages/auth/src/hashing.js b/packages/backend-core/src/hashing.js similarity index 100% rename from packages/auth/src/hashing.js rename to packages/backend-core/src/hashing.js diff --git a/packages/auth/src/helpers.js b/packages/backend-core/src/helpers.js similarity index 100% rename from packages/auth/src/helpers.js rename to packages/backend-core/src/helpers.js diff --git a/packages/backend-core/src/index.js b/packages/backend-core/src/index.js new file mode 100644 index 0000000000..cd3a3f5c97 --- /dev/null +++ b/packages/backend-core/src/index.js @@ -0,0 +1,17 @@ +const { setDB } = require("./db") + +module.exports = { + init(pouch) { + setDB(pouch) + }, + // some default exports from the library, however these ideally shouldn't + // be used, instead the syntax require("@budibase/backend-core/db") should be used + StaticDatabases: require("./db/utils").StaticDatabases, + db: require("../db"), + redis: require("../redis"), + objectStore: require("../objectStore"), + utils: require("../utils"), + cache: require("../cache"), + auth: require("../auth"), + constants: require("../constants"), +} diff --git a/packages/auth/src/middleware/appTenancy.js b/packages/backend-core/src/middleware/appTenancy.js similarity index 100% rename from packages/auth/src/middleware/appTenancy.js rename to packages/backend-core/src/middleware/appTenancy.js diff --git a/packages/auth/src/middleware/auditLog.js b/packages/backend-core/src/middleware/auditLog.js similarity index 100% rename from packages/auth/src/middleware/auditLog.js rename to packages/backend-core/src/middleware/auditLog.js diff --git a/packages/auth/src/middleware/authenticated.js b/packages/backend-core/src/middleware/authenticated.js similarity index 92% rename from packages/auth/src/middleware/authenticated.js rename to packages/backend-core/src/middleware/authenticated.js index f0fb6e21c5..87bd4d35ce 100644 --- a/packages/auth/src/middleware/authenticated.js +++ b/packages/backend-core/src/middleware/authenticated.js @@ -1,5 +1,5 @@ const { Cookies, Headers } = require("../constants") -const { getCookie, clearCookie } = require("../utils") +const { getCookie, clearCookie, openJwt } = require("../utils") const { getUser } = require("../cache/user") const { getSession, updateSessionTTL } = require("../security/sessions") const { buildMatcherRegex, matches } = require("./matchers") @@ -35,8 +35,9 @@ module.exports = ( publicEndpoint = true } try { - // check the actual user is authenticated first - const authCookie = getCookie(ctx, Cookies.Auth) + // check the actual user is authenticated first, try header or cookie + const headerToken = ctx.request.headers[Headers.TOKEN] + const authCookie = getCookie(ctx, Cookies.Auth) || openJwt(headerToken) let authenticated = false, user = null, internal = false diff --git a/packages/auth/src/middleware/index.js b/packages/backend-core/src/middleware/index.js similarity index 100% rename from packages/auth/src/middleware/index.js rename to packages/backend-core/src/middleware/index.js diff --git a/packages/auth/src/middleware/matchers.js b/packages/backend-core/src/middleware/matchers.js similarity index 100% rename from packages/auth/src/middleware/matchers.js rename to packages/backend-core/src/middleware/matchers.js diff --git a/packages/auth/src/middleware/passport/google.js b/packages/backend-core/src/middleware/passport/google.js similarity index 100% rename from packages/auth/src/middleware/passport/google.js rename to packages/backend-core/src/middleware/passport/google.js diff --git a/packages/auth/src/middleware/passport/jwt.js b/packages/backend-core/src/middleware/passport/jwt.js similarity index 100% rename from packages/auth/src/middleware/passport/jwt.js rename to packages/backend-core/src/middleware/passport/jwt.js diff --git a/packages/auth/src/middleware/passport/local.js b/packages/backend-core/src/middleware/passport/local.js similarity index 100% rename from packages/auth/src/middleware/passport/local.js rename to packages/backend-core/src/middleware/passport/local.js diff --git a/packages/auth/src/middleware/passport/oidc.js b/packages/backend-core/src/middleware/passport/oidc.js similarity index 100% rename from packages/auth/src/middleware/passport/oidc.js rename to packages/backend-core/src/middleware/passport/oidc.js diff --git a/packages/auth/src/middleware/passport/tests/google.spec.js b/packages/backend-core/src/middleware/passport/tests/google.spec.js similarity index 100% rename from packages/auth/src/middleware/passport/tests/google.spec.js rename to packages/backend-core/src/middleware/passport/tests/google.spec.js diff --git a/packages/auth/src/middleware/passport/tests/oidc.spec.js b/packages/backend-core/src/middleware/passport/tests/oidc.spec.js similarity index 100% rename from packages/auth/src/middleware/passport/tests/oidc.spec.js rename to packages/backend-core/src/middleware/passport/tests/oidc.spec.js diff --git a/packages/auth/src/middleware/passport/tests/third-party-common.spec.js b/packages/backend-core/src/middleware/passport/tests/third-party-common.spec.js similarity index 100% rename from packages/auth/src/middleware/passport/tests/third-party-common.spec.js rename to packages/backend-core/src/middleware/passport/tests/third-party-common.spec.js diff --git a/packages/auth/src/middleware/passport/tests/utilities/mock-data.js b/packages/backend-core/src/middleware/passport/tests/utilities/mock-data.js similarity index 100% rename from packages/auth/src/middleware/passport/tests/utilities/mock-data.js rename to packages/backend-core/src/middleware/passport/tests/utilities/mock-data.js diff --git a/packages/auth/src/middleware/passport/third-party-common.js b/packages/backend-core/src/middleware/passport/third-party-common.js similarity index 100% rename from packages/auth/src/middleware/passport/third-party-common.js rename to packages/backend-core/src/middleware/passport/third-party-common.js diff --git a/packages/auth/src/middleware/passport/utils.js b/packages/backend-core/src/middleware/passport/utils.js similarity index 100% rename from packages/auth/src/middleware/passport/utils.js rename to packages/backend-core/src/middleware/passport/utils.js diff --git a/packages/auth/src/middleware/tenancy.js b/packages/backend-core/src/middleware/tenancy.js similarity index 100% rename from packages/auth/src/middleware/tenancy.js rename to packages/backend-core/src/middleware/tenancy.js diff --git a/packages/auth/src/migrations/index.js b/packages/backend-core/src/migrations/index.js similarity index 100% rename from packages/auth/src/migrations/index.js rename to packages/backend-core/src/migrations/index.js diff --git a/packages/auth/src/migrations/tests/__snapshots__/index.spec.js.snap b/packages/backend-core/src/migrations/tests/__snapshots__/index.spec.js.snap similarity index 100% rename from packages/auth/src/migrations/tests/__snapshots__/index.spec.js.snap rename to packages/backend-core/src/migrations/tests/__snapshots__/index.spec.js.snap diff --git a/packages/auth/src/migrations/tests/index.spec.js b/packages/backend-core/src/migrations/tests/index.spec.js similarity index 100% rename from packages/auth/src/migrations/tests/index.spec.js rename to packages/backend-core/src/migrations/tests/index.spec.js diff --git a/packages/auth/src/objectStore/index.js b/packages/backend-core/src/objectStore/index.js similarity index 91% rename from packages/auth/src/objectStore/index.js rename to packages/backend-core/src/objectStore/index.js index 87b67d464e..b5d8475cee 100644 --- a/packages/auth/src/objectStore/index.js +++ b/packages/backend-core/src/objectStore/index.js @@ -206,6 +206,34 @@ exports.retrieveToTmp = async (bucketName, filepath) => { return outputPath } +/** + * Delete a single file. + */ +exports.deleteFile = async (bucketName, filepath) => { + const objectStore = exports.ObjectStore(bucketName) + await exports.makeSureBucketExists(objectStore, bucketName) + const params = { + Bucket: bucketName, + Key: filepath, + } + return objectStore.deleteObject(params) +} + +exports.deleteFiles = async (bucketName, filepaths) => { + const objectStore = exports.ObjectStore(bucketName) + await exports.makeSureBucketExists(objectStore, bucketName) + const params = { + Bucket: bucketName, + Delete: { + Objects: filepaths.map(path => ({ Key: path })), + }, + } + return objectStore.deleteObjects(params).promise() +} + +/** + * Delete a path, including everything within. + */ exports.deleteFolder = async (bucketName, folder) => { bucketName = sanitizeBucket(bucketName) folder = sanitizeKey(folder) diff --git a/packages/auth/src/objectStore/utils.js b/packages/backend-core/src/objectStore/utils.js similarity index 100% rename from packages/auth/src/objectStore/utils.js rename to packages/backend-core/src/objectStore/utils.js diff --git a/packages/auth/src/redis/authRedis.js b/packages/backend-core/src/redis/authRedis.js similarity index 100% rename from packages/auth/src/redis/authRedis.js rename to packages/backend-core/src/redis/authRedis.js diff --git a/packages/auth/src/redis/index.js b/packages/backend-core/src/redis/index.js similarity index 100% rename from packages/auth/src/redis/index.js rename to packages/backend-core/src/redis/index.js diff --git a/packages/auth/src/redis/utils.js b/packages/backend-core/src/redis/utils.js similarity index 98% rename from packages/auth/src/redis/utils.js rename to packages/backend-core/src/redis/utils.js index 466b117e96..3461d2a511 100644 --- a/packages/auth/src/redis/utils.js +++ b/packages/backend-core/src/redis/utils.js @@ -16,6 +16,7 @@ exports.Databases = { USER_CACHE: "users", FLAGS: "flags", APP_METADATA: "appMetadata", + QUERY_VARS: "queryVars", } exports.SEPARATOR = SEPARATOR diff --git a/packages/auth/src/security/permissions.js b/packages/backend-core/src/security/permissions.js similarity index 100% rename from packages/auth/src/security/permissions.js rename to packages/backend-core/src/security/permissions.js diff --git a/packages/auth/src/security/roles.js b/packages/backend-core/src/security/roles.js similarity index 100% rename from packages/auth/src/security/roles.js rename to packages/backend-core/src/security/roles.js diff --git a/packages/auth/src/security/sessions.js b/packages/backend-core/src/security/sessions.js similarity index 100% rename from packages/auth/src/security/sessions.js rename to packages/backend-core/src/security/sessions.js diff --git a/packages/auth/src/tenancy/FunctionContext.js b/packages/backend-core/src/tenancy/FunctionContext.js similarity index 100% rename from packages/auth/src/tenancy/FunctionContext.js rename to packages/backend-core/src/tenancy/FunctionContext.js diff --git a/packages/auth/src/tenancy/context.js b/packages/backend-core/src/tenancy/context.js similarity index 100% rename from packages/auth/src/tenancy/context.js rename to packages/backend-core/src/tenancy/context.js diff --git a/packages/auth/src/tenancy/deprovision.js b/packages/backend-core/src/tenancy/deprovision.js similarity index 100% rename from packages/auth/src/tenancy/deprovision.js rename to packages/backend-core/src/tenancy/deprovision.js diff --git a/packages/auth/src/tenancy/index.js b/packages/backend-core/src/tenancy/index.js similarity index 100% rename from packages/auth/src/tenancy/index.js rename to packages/backend-core/src/tenancy/index.js diff --git a/packages/auth/src/tenancy/tenancy.js b/packages/backend-core/src/tenancy/tenancy.js similarity index 100% rename from packages/auth/src/tenancy/tenancy.js rename to packages/backend-core/src/tenancy/tenancy.js diff --git a/packages/auth/src/tests/utilities/db.js b/packages/backend-core/src/tests/utilities/db.js similarity index 100% rename from packages/auth/src/tests/utilities/db.js rename to packages/backend-core/src/tests/utilities/db.js diff --git a/packages/auth/src/tests/utilities/dbConfig.js b/packages/backend-core/src/tests/utilities/dbConfig.js similarity index 100% rename from packages/auth/src/tests/utilities/dbConfig.js rename to packages/backend-core/src/tests/utilities/dbConfig.js diff --git a/packages/auth/src/utils.js b/packages/backend-core/src/utils.js similarity index 94% rename from packages/auth/src/utils.js rename to packages/backend-core/src/utils.js index b8fa7b9588..85dd32946f 100644 --- a/packages/auth/src/utils.js +++ b/packages/backend-core/src/utils.js @@ -63,6 +63,17 @@ exports.getAppId = ctx => { return appId } +/** + * opens the contents of the specified encrypted JWT. + * @return {object} the contents of the token. + */ +exports.openJwt = token => { + if (!token) { + return token + } + return jwt.verify(token, options.secretOrKey) +} + /** * Get a cookie from context, and decrypt if necessary. * @param {object} ctx The request which is to be manipulated. @@ -75,7 +86,7 @@ exports.getCookie = (ctx, name) => { return cookie } - return jwt.verify(cookie, options.secretOrKey) + return exports.openJwt(cookie) } /** @@ -85,7 +96,12 @@ exports.getCookie = (ctx, name) => { * @param {string|object} value The value of cookie which will be set. * @param {object} opts options like whether to sign. */ -exports.setCookie = (ctx, value, name = "builder", opts = { sign: true }) => { +exports.setCookie = ( + ctx, + value, + name = "builder", + opts = { sign: true, requestDomain: false } +) => { if (value && opts && opts.sign) { value = jwt.sign(value, options.secretOrKey) } @@ -97,7 +113,7 @@ exports.setCookie = (ctx, value, name = "builder", opts = { sign: true }) => { overwrite: true, } - if (environment.COOKIE_DOMAIN) { + if (environment.COOKIE_DOMAIN && !opts.requestDomain) { config.domain = environment.COOKIE_DOMAIN } diff --git a/packages/auth/tenancy.js b/packages/backend-core/tenancy.js similarity index 100% rename from packages/auth/tenancy.js rename to packages/backend-core/tenancy.js diff --git a/packages/backend-core/utils.js b/packages/backend-core/utils.js new file mode 100644 index 0000000000..2ef920e103 --- /dev/null +++ b/packages/backend-core/utils.js @@ -0,0 +1,4 @@ +module.exports = { + ...require("./src/utils"), + ...require("./src/hashing"), +} diff --git a/packages/auth/yarn.lock b/packages/backend-core/yarn.lock similarity index 100% rename from packages/auth/yarn.lock rename to packages/backend-core/yarn.lock diff --git a/packages/bbui/package.json b/packages/bbui/package.json index 0ae4471dd9..9515461230 100644 --- a/packages/bbui/package.json +++ b/packages/bbui/package.json @@ -1,7 +1,7 @@ { "name": "@budibase/bbui", "description": "A UI solution used in the different Budibase projects.", - "version": "1.0.27-alpha.0", + "version": "1.0.44-alpha.1", "license": "MPL-2.0", "svelte": "src/index.js", "module": "dist/bbui.es.js", diff --git a/packages/bbui/src/Drawer/Drawer.svelte b/packages/bbui/src/Drawer/Drawer.svelte index 5e3de88e61..4bc8a69445 100644 --- a/packages/bbui/src/Drawer/Drawer.svelte +++ b/packages/bbui/src/Drawer/Drawer.svelte @@ -21,16 +21,8 @@ } visible = false } - - function handleKey(e) { - if (visible && e.key === "Escape") { - hide() - } - } - - {#if visible}
diff --git a/packages/bbui/src/Table/BoldRenderer.svelte b/packages/bbui/src/Table/BoldRenderer.svelte new file mode 100644 index 0000000000..ea882d538e --- /dev/null +++ b/packages/bbui/src/Table/BoldRenderer.svelte @@ -0,0 +1,11 @@ + + +
{value}
+ + diff --git a/packages/bbui/src/Table/CodeRenderer.svelte b/packages/bbui/src/Table/CodeRenderer.svelte new file mode 100644 index 0000000000..a75bec663c --- /dev/null +++ b/packages/bbui/src/Table/CodeRenderer.svelte @@ -0,0 +1,5 @@ + + +{value} diff --git a/packages/bbui/src/Table/Table.svelte b/packages/bbui/src/Table/Table.svelte index bcd84e7112..09ade22627 100644 --- a/packages/bbui/src/Table/Table.svelte +++ b/packages/bbui/src/Table/Table.svelte @@ -4,6 +4,7 @@ import CellRenderer from "./CellRenderer.svelte" import SelectEditRenderer from "./SelectEditRenderer.svelte" import { cloneDeep } from "lodash" + import { deepGet } from "../utils/helpers" /** * The expected schema is our normal couch schemas for our tables. @@ -318,7 +319,7 @@ {customRenderers} {row} schema={schema[field]} - value={row[field]} + value={deepGet(row, field)} on:clickrelationship > diff --git a/packages/bbui/src/Tabs/Tabs.svelte b/packages/bbui/src/Tabs/Tabs.svelte index c68f715de0..40e02058c1 100644 --- a/packages/bbui/src/Tabs/Tabs.svelte +++ b/packages/bbui/src/Tabs/Tabs.svelte @@ -10,6 +10,8 @@ export let noHorizPadding = false export let quiet = false export let emphasized = false + // overlay content from the tab bar onto tabs e.g. for a dropdown + export let onTop = false let thisSelected = undefined @@ -78,6 +80,7 @@ 'spectrum-Tabs--quiet'} spectrum-Tabs--{vertical ? 'vertical' : 'horizontal'}" + class:onTop > {#if $tab.info} @@ -98,7 +101,9 @@ .quiet { border-bottom: none !important; } - + .onTop { + z-index: 20; + } .spectrum-Tabs { padding-left: var(--spacing-xl); padding-right: var(--spacing-xl); diff --git a/packages/bbui/src/index.js b/packages/bbui/src/index.js index c30047f73f..16f069f4e7 100644 --- a/packages/bbui/src/index.js +++ b/packages/bbui/src/index.js @@ -61,6 +61,10 @@ export { default as ColorPicker } from "./ColorPicker/ColorPicker.svelte" export { default as InlineAlert } from "./InlineAlert/InlineAlert.svelte" export { default as Banner } from "./Banner/Banner.svelte" +// Renderers +export { default as BoldRenderer } from "./Table/BoldRenderer.svelte" +export { default as CodeRenderer } from "./Table/CodeRenderer.svelte" + // Typography export { default as Body } from "./Typography/Body.svelte" export { default as Heading } from "./Typography/Heading.svelte" @@ -77,3 +81,6 @@ export { default as clickOutside } from "./Actions/click_outside" // Stores export { notifications, createNotificationStore } from "./Stores/notifications" + +// Utils +export * from "./utils/helpers" diff --git a/packages/bbui/src/utils/helpers.js b/packages/bbui/src/utils/helpers.js index 83d305d573..6cf432f356 100644 --- a/packages/bbui/src/utils/helpers.js +++ b/packages/bbui/src/utils/helpers.js @@ -6,3 +6,61 @@ export const generateID = () => { } export const capitalise = s => s.substring(0, 1).toUpperCase() + s.substring(1) + +/** + * Gets a key within an object. The key supports dot syntax for retrieving deep + * fields - e.g. "a.b.c". + * Exact matches of keys with dots in them take precedence over nested keys of + * the same path - e.g. getting "a.b" from { "a.b": "foo", a: { b: "bar" } } + * will return "foo" over "bar". + * @param obj the object + * @param key the key + * @return {*|null} the value or null if a value was not found for this key + */ +export const deepGet = (obj, key) => { + if (!obj || !key) { + return null + } + if (Object.prototype.hasOwnProperty.call(obj, key)) { + return obj[key] + } + const split = key.split(".") + for (let i = 0; i < split.length; i++) { + obj = obj?.[split[i]] + } + return obj +} + +/** + * Sets a key within an object. The key supports dot syntax for retrieving deep + * fields - e.g. "a.b.c". + * Exact matches of keys with dots in them take precedence over nested keys of + * the same path - e.g. setting "a.b" of { "a.b": "foo", a: { b: "bar" } } + * will override the value "foo" rather than "bar". + * If a deep path is specified and the parent keys don't exist then these will + * be created. + * @param obj the object + * @param key the key + * @param value the value + */ +export const deepSet = (obj, key, value) => { + if (!obj || !key) { + return + } + if (Object.prototype.hasOwnProperty.call(obj, key)) { + obj[key] = value + return + } + const split = key.split(".") + for (let i = 0; i < split.length - 1; i++) { + const nextKey = split[i] + if (obj && obj[nextKey] == null) { + obj[nextKey] = {} + } + obj = obj?.[nextKey] + } + if (!obj) { + return + } + obj[split[split.length - 1]] = value +} diff --git a/packages/bbui/yarn.lock b/packages/bbui/yarn.lock index a102e6c148..42c88af5a4 100644 --- a/packages/bbui/yarn.lock +++ b/packages/bbui/yarn.lock @@ -2076,9 +2076,9 @@ postcss@^7.0.0, postcss@^7.0.1, postcss@^7.0.27: supports-color "^6.1.0" postcss@^8.2.9: - version "8.2.10" - resolved "https://registry.yarnpkg.com/postcss/-/postcss-8.2.10.tgz#ca7a042aa8aff494b334d0ff3e9e77079f6f702b" - integrity sha512-b/h7CPV7QEdrqIxtAf2j31U5ef05uBDuvoXv6L51Q4rcS1jdlXAVKJv+atCFdUXYl9dyTHGyoMzIepwowRJjFw== + version "8.2.13" + resolved "https://registry.yarnpkg.com/postcss/-/postcss-8.2.13.tgz#dbe043e26e3c068e45113b1ed6375d2d37e2129f" + integrity sha512-FCE5xLH+hjbzRdpbRb1IMCvPv9yZx2QnDarBEYSN0N0HYk+TcXsEhwdFcFb+SRWOKzKGErhIEbBK2ogyLdTtfQ== dependencies: colorette "^1.2.2" nanoid "^3.1.22" diff --git a/packages/builder/package.json b/packages/builder/package.json index 645289de8e..90b95c9b32 100644 --- a/packages/builder/package.json +++ b/packages/builder/package.json @@ -1,6 +1,6 @@ { "name": "@budibase/builder", - "version": "1.0.27-alpha.0", + "version": "1.0.44-alpha.1", "license": "GPL-3.0", "private": true, "scripts": { @@ -65,10 +65,10 @@ } }, "dependencies": { - "@budibase/bbui": "^1.0.27-alpha.0", - "@budibase/client": "^1.0.27-alpha.0", + "@budibase/bbui": "^1.0.44-alpha.1", + "@budibase/client": "^1.0.44-alpha.1", "@budibase/colorpicker": "1.1.2", - "@budibase/string-templates": "^1.0.27-alpha.0", + "@budibase/string-templates": "^1.0.44-alpha.1", "@sentry/browser": "5.19.1", "@spectrum-css/page": "^3.0.1", "@spectrum-css/vars": "^3.0.1", diff --git a/packages/builder/src/builderStore/api.js b/packages/builder/src/builderStore/api.js index 4bcb9b74c6..897d3a74db 100644 --- a/packages/builder/src/builderStore/api.js +++ b/packages/builder/src/builderStore/api.js @@ -13,6 +13,9 @@ const apiCall = headers, }) if (resp.status === 403) { + if (url.includes("/api/templates")) { + return { json: () => [] } + } removeCookie(Cookies.Auth) // reload after removing cookie, go to login if (!url.includes("self") && !url.includes("login")) { diff --git a/packages/builder/src/builderStore/storeUtils.js b/packages/builder/src/builderStore/componentUtils.js similarity index 80% rename from packages/builder/src/builderStore/storeUtils.js rename to packages/builder/src/builderStore/componentUtils.js index e25949000f..04a87998fe 100644 --- a/packages/builder/src/builderStore/storeUtils.js +++ b/packages/builder/src/builderStore/componentUtils.js @@ -127,18 +127,37 @@ const searchComponentTree = (rootComponent, matchComponent) => { } /** - * Searches a component's definition for a setting matching a certin predicate. + * Searches a component's definition for a setting matching a certain predicate. + * These settings are cached because they cannot change at run time. */ +let componentSettingCache = {} export const getComponentSettings = componentType => { - const def = store.actions.components.getDefinition(componentType) - if (!def) { + if (!componentType) { return [] } - let settings = def.settings?.filter(setting => !setting.section) ?? [] - def.settings - ?.filter(setting => setting.section) - .forEach(section => { - settings = settings.concat(section.settings || []) - }) + + // Ensure whole component name is used + if (!componentType.startsWith("@budibase")) { + componentType = `@budibase/standard-components/${componentType}` + } + + // Check if we have cached this type already + if (componentSettingCache[componentType]) { + return componentSettingCache[componentType] + } + + // Otherwise get the settings and cache them + const def = store.actions.components.getDefinition(componentType) + let settings = [] + if (def) { + settings = def.settings?.filter(setting => !setting.section) ?? [] + def.settings + ?.filter(setting => setting.section) + .forEach(section => { + settings = settings.concat(section.settings || []) + }) + } + componentSettingCache[componentType] = settings + return settings } diff --git a/packages/builder/src/builderStore/cookies.js b/packages/builder/src/builderStore/cookies.js index a84f1a4f20..cb4e46ec86 100644 --- a/packages/builder/src/builderStore/cookies.js +++ b/packages/builder/src/builderStore/cookies.js @@ -1,16 +1,26 @@ export const Cookies = { Auth: "budibase:auth", CurrentApp: "budibase:currentapp", + ReturnUrl: "budibase:returnurl", +} + +export function setCookie(name, value) { + if (getCookie(name)) { + removeCookie(name) + } + window.document.cookie = `${name}=${value}; Path=/;` } export function getCookie(cookieName) { - return document.cookie.split(";").some(cookie => { - return cookie.trim().startsWith(`${cookieName}=`) - }) + const value = `; ${document.cookie}` + const parts = value.split(`; ${cookieName}=`) + if (parts.length === 2) { + return parts[1].split(";").shift() + } } export function removeCookie(cookieName) { if (getCookie(cookieName)) { - document.cookie = `${cookieName}=; Max-Age=-99999999;` + document.cookie = `${cookieName}=; Max-Age=-99999999; Path=/;` } } diff --git a/packages/builder/src/builderStore/dataBinding.js b/packages/builder/src/builderStore/dataBinding.js index b36613fbc5..0f3cffc4fb 100644 --- a/packages/builder/src/builderStore/dataBinding.js +++ b/packages/builder/src/builderStore/dataBinding.js @@ -5,7 +5,7 @@ import { findComponent, findComponentPath, getComponentSettings, -} from "./storeUtils" +} from "./componentUtils" import { store } from "builderStore" import { queries as queriesStores, tables as tablesStore } from "stores/backend" import { @@ -15,6 +15,11 @@ import { encodeJSBinding, } from "@budibase/string-templates" import { TableNames } from "../constants" +import { + convertJSONSchemaToTableSchema, + getJSONArrayDatasourceSchema, +} from "./jsonUtils" +import ActionDefinitions from "components/design/PropertiesPanel/PropertyControls/ButtonActionEditor/manifest.json" // Regex to match all instances of template strings const CAPTURE_VAR_INSIDE_TEMPLATE = /{{([^}]+)}}/g @@ -186,6 +191,7 @@ const getProviderContextBindings = (asset, dataProviders) => { } let schema + let table let readablePrefix let runtimeSuffix = context.suffix @@ -209,7 +215,16 @@ const getProviderContextBindings = (asset, dataProviders) => { } const info = getSchemaForDatasource(asset, datasource) schema = info.schema - readablePrefix = info.table?.name + table = info.table + + // For JSON arrays, use the array name as the readable prefix. + // Otherwise use the table name + if (datasource.type === "jsonarray") { + const split = datasource.label.split(".") + readablePrefix = split[split.length - 1] + } else { + readablePrefix = info.table?.name + } } if (!schema) { return @@ -229,7 +244,8 @@ const getProviderContextBindings = (asset, dataProviders) => { const fieldSchema = schema[key] // Make safe runtime binding - const runtimeBinding = `${safeComponentId}.${makePropSafe(key)}` + const safeKey = key.split(".").map(makePropSafe).join(".") + const runtimeBinding = `${safeComponentId}.${safeKey}` // Optionally use a prefix with readable bindings let readableBinding = component._instanceName @@ -247,6 +263,8 @@ const getProviderContextBindings = (asset, dataProviders) => { // datasource options, based on bindable properties fieldSchema, providerId, + // Table ID is used by JSON fields to know what table the field is in + tableId: table?._id, }) }) }) @@ -339,6 +357,36 @@ const getUrlBindings = asset => { })) } +/** + * Gets all bindable properties exposed in a button actions flow up until + * the specified action ID. + */ +export const getButtonContextBindings = (actions, actionId) => { + // Get the steps leading up to this value + const index = actions?.findIndex(action => action.id === actionId) + if (index == null || index === -1) { + return [] + } + const prevActions = actions.slice(0, index) + + // Generate bindings for any steps which provide context + let bindings = [] + prevActions.forEach((action, idx) => { + const def = ActionDefinitions.actions.find( + x => x.name === action["##eventHandlerType"] + ) + if (def.context) { + def.context.forEach(contextValue => { + bindings.push({ + readableBinding: `Action ${idx + 1}.${contextValue.label}`, + runtimeBinding: `actions.${idx}.${contextValue.value}`, + }) + }) + } + }) + return bindings +} + /** * Gets a schema for a datasource object. */ @@ -347,16 +395,26 @@ export const getSchemaForDatasource = (asset, datasource, isForm = false) => { if (datasource) { const { type } = datasource + const tables = get(tablesStore).list - // Determine the source table from the datasource type + // Determine the entity which backs this datasource. + // "provider" datasources are those targeting another data provider if (type === "provider") { const component = findComponent(asset.props, datasource.providerId) const source = getDatasourceForProvider(asset, component) return getSchemaForDatasource(asset, source, isForm) - } else if (type === "query") { + } + + // "query" datasources are those targeting non-plus datasources or + // custom queries + else if (type === "query") { const queries = get(queriesStores).list table = queries.find(query => query._id === datasource._id) - } else if (type === "field") { + } + + // "field" datasources are array-like fields of rows, such as attachments + // or multi-select fields + else if (type === "field") { table = { name: datasource.fieldName } const { fieldType } = datasource if (fieldType === "attachment") { @@ -375,12 +433,22 @@ export const getSchemaForDatasource = (asset, datasource, isForm = false) => { }, } } - } else { - const tables = get(tablesStore).list + } + + // "jsonarray" datasources are arrays inside JSON fields + else if (type === "jsonarray") { + table = tables.find(table => table._id === datasource.tableId) + let tableSchema = table?.schema + schema = getJSONArrayDatasourceSchema(tableSchema, datasource) + } + + // Otherwise we assume we're targeting an internal table or a plus + // datasource, and we can treat it as a table with a schema + else { table = tables.find(table => table._id === datasource.tableId) } - // Determine the schema from the table if not already determined + // Determine the schema from the backing entity if not already determined if (table && !schema) { if (type === "view") { schema = cloneDeep(table.views?.[datasource.name]?.schema) @@ -397,6 +465,26 @@ export const getSchemaForDatasource = (asset, datasource, isForm = false) => { } } + // Check for any JSON fields so we can add any top level properties + if (schema) { + let jsonAdditions = {} + Object.keys(schema).forEach(fieldKey => { + const fieldSchema = schema[fieldKey] + if (fieldSchema?.type === "json") { + const jsonSchema = convertJSONSchemaToTableSchema(fieldSchema, { + squashObjects: true, + }) + Object.keys(jsonSchema).forEach(jsonKey => { + jsonAdditions[`${fieldKey}.${jsonKey}`] = { + type: jsonSchema[jsonKey].type, + nestedJSON: true, + } + }) + } + }) + schema = { ...schema, ...jsonAdditions } + } + // Add _id and _rev fields for certain types if (schema && !isForm && ["table", "link"].includes(datasource.type)) { schema["_id"] = { type: "string" } @@ -450,15 +538,58 @@ const buildFormSchema = component => { return schema } +/** + * Returns an array of the keys of any state variables which are set anywhere + * in the app. + */ +export const getAllStateVariables = () => { + // Get all component containing assets + let allAssets = [] + allAssets = allAssets.concat(get(store).layouts || []) + allAssets = allAssets.concat(get(store).screens || []) + + // Find all button action settings in all components + let eventSettings = [] + allAssets.forEach(asset => { + findAllMatchingComponents(asset.props, component => { + const settings = getComponentSettings(component._component) + settings + .filter(setting => setting.type === "event") + .forEach(setting => { + eventSettings.push(component[setting.key]) + }) + }) + }) + + // Extract all state keys from any "update state" actions in each setting + let bindingSet = new Set() + eventSettings.forEach(setting => { + if (!Array.isArray(setting)) { + return + } + setting.forEach(action => { + if ( + action["##eventHandlerType"] === "Update State" && + action.parameters?.type === "set" && + action.parameters?.key && + action.parameters?.value + ) { + bindingSet.add(action.parameters.key) + } + }) + }) + return Array.from(bindingSet) +} + /** * Recurses the input object to remove any instances of bindings. */ -export function removeBindings(obj) { +export const removeBindings = (obj, replacement = "Invalid binding") => { for (let [key, value] of Object.entries(obj)) { if (value && typeof value === "object") { - obj[key] = removeBindings(value) + obj[key] = removeBindings(value, replacement) } else if (typeof value === "string") { - obj[key] = value.replace(CAPTURE_HBS_TEMPLATE, "Invalid binding") + obj[key] = value.replace(CAPTURE_HBS_TEMPLATE, replacement) } } return obj @@ -468,8 +599,8 @@ export function removeBindings(obj) { * When converting from readable to runtime it can sometimes add too many square brackets, * this makes sure that doesn't happen. */ -function shouldReplaceBinding(currentValue, from, convertTo) { - if (!currentValue?.includes(from)) { +const shouldReplaceBinding = (currentValue, convertFrom, convertTo) => { + if (!currentValue?.includes(convertFrom)) { return false } if (convertTo === "readableBinding") { @@ -478,7 +609,7 @@ function shouldReplaceBinding(currentValue, from, convertTo) { // remove all the spaces, if the input is surrounded by spaces e.g. [ Auto ID ] then // this makes sure it is detected const noSpaces = currentValue.replace(/\s+/g, "") - const fromNoSpaces = from.replace(/\s+/g, "") + const fromNoSpaces = convertFrom.replace(/\s+/g, "") const invalids = [ `[${fromNoSpaces}]`, `"${fromNoSpaces}"`, @@ -487,14 +618,21 @@ function shouldReplaceBinding(currentValue, from, convertTo) { return !invalids.find(invalid => noSpaces?.includes(invalid)) } -function replaceBetween(string, start, end, replacement) { +/** + * Utility function which replaces a string between given indices. + */ +const replaceBetween = (string, start, end, replacement) => { return string.substring(0, start) + replacement + string.substring(end) } /** - * utility function for the readableToRuntimeBinding and runtimeToReadableBinding. + * Utility function for the readableToRuntimeBinding and runtimeToReadableBinding. */ -function bindingReplacement(bindableProperties, textWithBindings, convertTo) { +const bindingReplacement = ( + bindableProperties, + textWithBindings, + convertTo +) => { // Decide from base64 if using JS const isJS = isJSBinding(textWithBindings) if (isJS) { @@ -559,14 +697,17 @@ function bindingReplacement(bindableProperties, textWithBindings, convertTo) { * Extracts a component ID from a handlebars expression setting of * {{ literal [componentId] }} */ -function extractLiteralHandlebarsID(value) { +const extractLiteralHandlebarsID = value => { return value?.match(/{{\s*literal\s*\[+([^\]]+)].*}}/)?.[1] } /** * Converts a readable data binding into a runtime data binding */ -export function readableToRuntimeBinding(bindableProperties, textWithBindings) { +export const readableToRuntimeBinding = ( + bindableProperties, + textWithBindings +) => { return bindingReplacement( bindableProperties, textWithBindings, @@ -577,56 +718,13 @@ export function readableToRuntimeBinding(bindableProperties, textWithBindings) { /** * Converts a runtime data binding into a readable data binding */ -export function runtimeToReadableBinding(bindableProperties, textWithBindings) { +export const runtimeToReadableBinding = ( + bindableProperties, + textWithBindings +) => { return bindingReplacement( bindableProperties, textWithBindings, "readableBinding" ) } - -/** - * Returns an array of the keys of any state variables which are set anywhere - * in the app. - */ -export const getAllStateVariables = () => { - let allComponents = [] - - // Find all onClick settings in all layouts - get(store).layouts.forEach(layout => { - const components = findAllMatchingComponents( - layout.props, - c => c.onClick != null - ) - allComponents = allComponents.concat(components || []) - }) - - // Find all onClick settings in all screens - get(store).screens.forEach(screen => { - const components = findAllMatchingComponents( - screen.props, - c => c.onClick != null - ) - allComponents = allComponents.concat(components || []) - }) - - // Add state bindings for all state actions - let bindingSet = new Set() - allComponents.forEach(component => { - if (!Array.isArray(component.onClick)) { - return - } - component.onClick.forEach(action => { - if ( - action["##eventHandlerType"] === "Update State" && - action.parameters?.type === "set" && - action.parameters?.key && - action.parameters?.value - ) { - bindingSet.add(action.parameters.key) - } - }) - }) - - return Array.from(bindingSet) -} diff --git a/packages/builder/src/builderStore/datasource.js b/packages/builder/src/builderStore/datasource.js index 61cead2e16..cfdeeac23e 100644 --- a/packages/builder/src/builderStore/datasource.js +++ b/packages/builder/src/builderStore/datasource.js @@ -23,10 +23,10 @@ function prepareData(config) { return datasource } -export async function saveDatasource(config) { +export async function saveDatasource(config, skipFetch = false) { const datasource = prepareData(config) // Create datasource - const resp = await datasources.save(datasource, datasource.plus) + const resp = await datasources.save(datasource, !skipFetch && datasource.plus) // update the tables incase data source plus await tables.fetch() diff --git a/packages/builder/src/builderStore/index.js b/packages/builder/src/builderStore/index.js index f32dedd47e..23704556ad 100644 --- a/packages/builder/src/builderStore/index.js +++ b/packages/builder/src/builderStore/index.js @@ -4,7 +4,7 @@ import { getHostingStore } from "./store/hosting" import { getThemeStore } from "./store/theme" import { derived, writable } from "svelte/store" import { FrontendTypes, LAYOUT_NAMES } from "../constants" -import { findComponent } from "./storeUtils" +import { findComponent } from "./componentUtils" export const store = getFrontendStore() export const automationStore = getAutomationStore() diff --git a/packages/builder/src/builderStore/jsonUtils.js b/packages/builder/src/builderStore/jsonUtils.js new file mode 100644 index 0000000000..29bf2df34e --- /dev/null +++ b/packages/builder/src/builderStore/jsonUtils.js @@ -0,0 +1,121 @@ +/** + * Gets the schema for a datasource which is targeting a JSON array, including + * nested JSON arrays. The returned schema is a squashed, table-like schema + * which is fully compatible with the rest of the platform. + * @param tableSchema the full schema for the table this JSON field is in + * @param datasource the datasource configuration + */ +export const getJSONArrayDatasourceSchema = (tableSchema, datasource) => { + let jsonSchema = tableSchema + let keysToSchema = [] + + // If we are already deep inside a JSON field then we need to account + // for the keys that brought us here, so we can get the schema for the + // depth we're actually at + if (datasource.prefixKeys) { + keysToSchema = datasource.prefixKeys.concat(["schema"]) + } + + // We parse the label of the datasource to work out where we are inside + // the structure. We can use this to know which part of the schema + // is available underneath our current position. + keysToSchema = keysToSchema.concat(datasource.label.split(".").slice(2)) + + // Follow the JSON key path until we reach the schema for the level + // we are at + for (let i = 0; i < keysToSchema.length; i++) { + jsonSchema = jsonSchema?.[keysToSchema[i]] + if (jsonSchema?.schema) { + jsonSchema = jsonSchema.schema + } + } + + // We need to convert the JSON schema into a more typical looking table + // schema so that it works with the rest of the platform + return convertJSONSchemaToTableSchema(jsonSchema, { + squashObjects: true, + prefixKeys: keysToSchema, + }) +} + +/** + * Converts a JSON field schema (or sub-schema of a nested field) into a schema + * that looks like a typical table schema. + * @param jsonSchema the JSON field schema or sub-schema + * @param options + */ +export const convertJSONSchemaToTableSchema = (jsonSchema, options) => { + if (!jsonSchema) { + return null + } + + // Add default options + options = { squashObjects: false, prefixKeys: null, ...options } + + // Immediately strip the wrapper schema for objects, or wrap shallow values in + // a fake "value" schema + if (jsonSchema.schema) { + jsonSchema = jsonSchema.schema + } else { + jsonSchema = { + value: jsonSchema, + } + } + + // Extract all deep keys from the schema + const keys = extractJSONSchemaKeys(jsonSchema, options.squashObjects) + + // Form a full schema from all the deep schema keys + let schema = {} + keys.forEach(({ key, type }) => { + schema[key] = { type, name: key, prefixKeys: options.prefixKeys } + }) + return schema +} + +/** + * Recursively builds paths to all leaf fields in a JSON field schema structure, + * stopping when leaf nodes or arrays are reached. + * @param jsonSchema the JSON field schema or sub-schema + * @param squashObjects whether to recurse into objects or not + */ +const extractJSONSchemaKeys = (jsonSchema, squashObjects = false) => { + if (!jsonSchema || !Object.keys(jsonSchema).length) { + return [] + } + + // Iterate through every schema key + let keys = [] + Object.keys(jsonSchema).forEach(key => { + const type = jsonSchema[key].type + + // If we encounter an object, then only go deeper if we want to squash + // object paths + if (type === "json" && squashObjects) { + // Find all keys within this objects schema + const childKeys = extractJSONSchemaKeys( + jsonSchema[key].schema, + squashObjects + ) + + // Append child paths onto the current path to build the full path + keys = keys.concat( + childKeys.map(childKey => ({ + key: `${key}.${childKey.key}`, + type: childKey.type, + })) + ) + } + + // Otherwise add this as a lead node. + // We transform array types from "array" into "jsonarray" here to avoid + // confusion with the existing "array" type that represents a multi-select. + else { + keys.push({ + key, + type: type === "array" ? "jsonarray" : type, + }) + } + }) + return keys +} diff --git a/packages/builder/src/builderStore/schemaGenerator.js b/packages/builder/src/builderStore/schemaGenerator.js new file mode 100644 index 0000000000..33115fc997 --- /dev/null +++ b/packages/builder/src/builderStore/schemaGenerator.js @@ -0,0 +1,56 @@ +import { FIELDS } from "constants/backend" + +function baseConversion(type) { + if (type === "string") { + return { + type: FIELDS.STRING.type, + } + } else if (type === "boolean") { + return { + type: FIELDS.BOOLEAN.type, + } + } else if (type === "number") { + return { + type: FIELDS.NUMBER.type, + } + } +} + +function recurse(schemaLevel = {}, objectLevel) { + if (!objectLevel) { + return null + } + const baseType = typeof objectLevel + if (baseType !== "object") { + return baseConversion(baseType) + } + for (let [key, value] of Object.entries(objectLevel)) { + const type = typeof value + // check array first, since arrays are objects + if (Array.isArray(value)) { + const schema = recurse(schemaLevel[key], value[0]) + if (schema) { + schemaLevel[key] = { + type: FIELDS.ARRAY.type, + schema, + } + } + } else if (type === "object") { + const schema = recurse(schemaLevel[key], objectLevel[key]) + if (schema) { + schemaLevel[key] = schema + } + } else { + schemaLevel[key] = baseConversion(type) + } + } + if (!schemaLevel.type) { + return { type: FIELDS.JSON.type, schema: schemaLevel } + } else { + return schemaLevel + } +} + +export function generate(object) { + return recurse({}, object).schema +} diff --git a/packages/builder/src/builderStore/store/frontend.js b/packages/builder/src/builderStore/store/frontend.js index 9f1a20605f..fdfe450edf 100644 --- a/packages/builder/src/builderStore/store/frontend.js +++ b/packages/builder/src/builderStore/store/frontend.js @@ -26,7 +26,7 @@ import { findAllMatchingComponents, findComponent, getComponentSettings, -} from "../storeUtils" +} from "../componentUtils" import { uuid } from "../uuid" import { removeBindings } from "../dataBinding" @@ -329,12 +329,12 @@ export const getFrontendStore = () => { }, components: { select: component => { - if (!component) { + const asset = get(currentAsset) + if (!asset || !component) { return } // If this is the root component, select the asset instead - const asset = get(currentAsset) const parent = findComponentParent(asset.props, component._id) if (parent == null) { const state = get(store) @@ -537,7 +537,7 @@ export const getFrontendStore = () => { // immediately need to remove bindings, currently these aren't valid when pasted if (!cut && !preserveBindings) { - state.componentToPaste = removeBindings(state.componentToPaste) + state.componentToPaste = removeBindings(state.componentToPaste, "") } // Clone the component to paste diff --git a/packages/builder/src/builderStore/store/screenTemplates/utils/commonComponents.js b/packages/builder/src/builderStore/store/screenTemplates/utils/commonComponents.js index 5b3bc041ff..ae45b4f25d 100644 --- a/packages/builder/src/builderStore/store/screenTemplates/utils/commonComponents.js +++ b/packages/builder/src/builderStore/store/screenTemplates/utils/commonComponents.js @@ -137,6 +137,7 @@ const fieldTypeToComponentMap = { datetime: "datetimefield", attachment: "attachmentfield", link: "relationshipfield", + json: "jsonfield", } export function makeDatasourceFormComponents(datasource) { @@ -146,7 +147,7 @@ export function makeDatasourceFormComponents(datasource) { fields.forEach(field => { const fieldSchema = schema[field] // skip autocolumns - if (fieldSchema.autocolumn) { + if (fieldSchema.autocolumn || fieldSchema.nestedJSON) { return } const fieldType = diff --git a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte index 7f379ba138..fe94b7e63f 100644 --- a/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte +++ b/packages/builder/src/components/automation/AutomationBuilder/FlowChart/FlowItem.svelte @@ -9,6 +9,7 @@ Modal, Button, StatusLight, + ActionButton, } from "@budibase/bbui" import AutomationBlockSetup from "../../SetupPanel/AutomationBlockSetup.svelte" import CreateWebhookModal from "components/automation/Shared/CreateWebhookModal.svelte" @@ -27,7 +28,7 @@ let blockComplete $: testResult = $automationStore.selectedAutomation.testResults?.steps.filter( - step => step.stepId === block.stepId + step => (block.id ? step.id === block.id : step.stepId === block.stepId) ) $: isTrigger = block.type === "TRIGGER" @@ -119,19 +120,13 @@
-
{ - setupToggled = !setupToggled - }} - class="center-items" + (setupToggled = !setupToggled)} + quiet + icon={setupToggled ? "ChevronDown" : "ChevronRight"} > - {#if setupToggled} - - {:else} - - {/if} Setup -
+ {#if !isTrigger}
deleteStep()}> @@ -187,6 +182,7 @@ .splitHeader { display: flex; justify-content: space-between; + align-items: center; } .iconAlign { padding: 0 0 0 var(--spacing-m); diff --git a/packages/builder/src/components/automation/SetupPanel/AutomationBlockSetup.svelte b/packages/builder/src/components/automation/SetupPanel/AutomationBlockSetup.svelte index d648b3f989..928897d6f5 100644 --- a/packages/builder/src/components/automation/SetupPanel/AutomationBlockSetup.svelte +++ b/packages/builder/src/components/automation/SetupPanel/AutomationBlockSetup.svelte @@ -10,6 +10,7 @@ ActionButton, Drawer, Modal, + Detail, } from "@budibase/bbui" import CreateWebhookModal from "components/automation/Shared/CreateWebhookModal.svelte" @@ -37,6 +38,7 @@ let drawer let tempFilters = lookForFilters(schemaProperties) || [] let fillWidth = true + let codeBindingOpen = false $: stepId = block.stepId $: bindings = getAvailableBindings( @@ -233,7 +235,16 @@ onChange(e, key)} value={inputData[key]} /> {:else if value.customType === "code"} - 
{JSON.stringify(bindings, null, 2)}
+ (codeBindingOpen = !codeBindingOpen)} + quiet + icon={codeBindingOpen ? "ChevronDown" : "ChevronRight"} + > + Bindings + + {#if codeBindingOpen} + 
{JSON.stringify(bindings, null, 2)}
+ {/if} { diff --git a/packages/builder/src/components/backend/DataTable/RowFieldControl.svelte b/packages/builder/src/components/backend/DataTable/RowFieldControl.svelte index 25ad67b52e..0d9ca3644b 100644 --- a/packages/builder/src/components/backend/DataTable/RowFieldControl.svelte +++ b/packages/builder/src/components/backend/DataTable/RowFieldControl.svelte @@ -6,16 +6,20 @@ Toggle, TextArea, Multiselect, + Label, } from "@budibase/bbui" import Dropzone from "components/common/Dropzone.svelte" import { capitalise } from "helpers" import LinkedRowSelector from "components/common/LinkedRowSelector.svelte" + import Editor from "../../integration/QueryEditor.svelte" export let defaultValue export let meta export let value = defaultValue || (meta.type === "boolean" ? false : "") export let readonly + $: stringVal = + typeof value === "object" ? JSON.stringify(value, null, 2) : value $: type = meta?.type $: label = meta.name ? capitalise(meta.name) : "" @@ -40,6 +44,14 @@ {:else if type === "longform"}