diff --git a/.github/workflows/README.md b/.github/workflows/README.md
index c33665c964..f77323d85a 100644
--- a/.github/workflows/README.md
+++ b/.github/workflows/README.md
@@ -119,6 +119,8 @@ This job is responsible for deploying to our production, cloud kubernetes enviro
## Pro
+| **NOTE**: When developing for both pro / budibase repositories, your branch names need to match, or else the correct pro doesn't get run within your CI job.
+
### Installing Pro
The pro package is always installed from source in our CI jobs.
@@ -132,7 +134,7 @@ This is done to prevent pro needing to be published prior to CI runs in budiabse
- backend-core lives in the monorepo, so it can't be released independently to be used in pro
- therefore the only option is to pull pro from source and release it as a part of the monorepo release, as if it were a mono package
-The install is performed using the same steps as local development, via the `yarn bootstrap` command, see the [Contributing Guide#Pro](../CONTRIBUTING.md#pro)
+The install is performed using the same steps as local development, via the `yarn bootstrap` command, see the [Contributing Guide#Pro](../../docs/CONTRIBUTING.md#pro)
The branch to install pro from can vary depending on ref of the commit that triggered the budibase CI job. This is done to enable branches which have changes in both the monorepo and the pro repo to have their CI pass successfully.
diff --git a/README.md b/README.md
index ae149f7347..1dec1737da 100644
--- a/README.md
+++ b/README.md
@@ -169,7 +169,7 @@ If you have a question or would like to talk with other Budibase users and join
## ❗ Code of conduct
-Budibase is dedicated to providing a welcoming, diverse, and harrassment-free experience for everyone. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/.github/CODE_OF_CONDUCT.md). Please read it.
+Budibase is dedicated to providing a welcoming, diverse, and harrassment-free experience for everyone. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/docs/CODE_OF_CONDUCT.md). Please read it.
diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md
index 531ed05749..fb0848596c 100644
--- a/docs/CONTRIBUTING.md
+++ b/docs/CONTRIBUTING.md
@@ -4,10 +4,10 @@ From opening a bug report to creating a pull request: every contribution is appr
## Table of contents
-- [Quick start](#quick-start)
-- [Status](#status)
-- [What's included](#whats-included)
-- [Bugs and feature requests](#bugs-and-feature-requests)
+- [Where to start](#not-sure-where-to-start)
+- [Contributor Licence Agreement](#contributor-license-agreement-cla)
+- [Glossary of Terms](#glossary-of-terms)
+- [Contributing to Budibase](#contributing-to-budibase)
## Not Sure Where to Start?
@@ -32,6 +32,9 @@ All contributors must sign an [Individual Contributor License Agreement](https:/
If contributing on behalf of your company, your company must sign a [Corporate Contributor License Agreement](https://github.com/budibase/budibase/blob/next/.github/cla/corporate-cla.md). If so, please contact us via community@budibase.com.
+If for any reason, your first contribution is in a PR created by other contributor, please just add a comment to the PR
+with the following text to agree our CLA: "I have read the CLA Document and I hereby sign the CLA".
+
## Glossary of Terms
To understand the budibase API, it can be helpful to understand the top level entities that make up Budibase.
@@ -162,7 +165,10 @@ When you are running locally, budibase stores data on disk using docker volumes.
### Development Modes
-A combination of environment variables controls the mode budibase runs in.
+A combination of environment variables controls the mode budibase runs in.
+
+| **NOTE**: You need to clean your browser cookies when you change between different modes.
+
Yarn commands can be used to mimic the different modes as described in the sections below:
#### Self Hosted
@@ -189,7 +195,7 @@ To enable this mode, use:
yarn mode:account
```
### CI
- An overview of the CI pipelines can be found [here](./workflows/README.md)
+ An overview of the CI pipelines can be found [here](../.github/workflows/README.md)
### Pro
diff --git a/docs/DEV-SETUP-MACOSX.md b/docs/DEV-SETUP-MACOSX.md
index 5606fd0d10..c5990e58da 100644
--- a/docs/DEV-SETUP-MACOSX.md
+++ b/docs/DEV-SETUP-MACOSX.md
@@ -4,6 +4,11 @@
Install instructions [here](https://brew.sh/)
+| **NOTE**: If you are working on a M1 Apple Silicon which is running Z shell, you could need to add
+`eval $(/opt/homebrew/bin/brew shellenv)` line to your `.zshrc`. This will make your zsh to find the apps you install
+through brew.
+
+
### Install Node
Budibase requires a recent version of node (14+):
@@ -51,4 +56,7 @@ So this command will actually run the application in dev mode. It creates .env f
The dev version will be available on port 10000 i.e.
-http://127.0.0.1:10000/builder/admin
\ No newline at end of file
+http://127.0.0.1:10000/builder/admin
+
+| **NOTE**: If you are working on a M1 Apple Silicon, you will need to uncomment `# platform: linux/amd64` line in
+[hosting/docker-compose-dev.yaml](../hosting/docker-compose.dev.yaml)
\ No newline at end of file
diff --git a/i18n/README.es.md b/i18n/README.es.md
index 7245dc8656..21eb8caef7 100644
--- a/i18n/README.es.md
+++ b/i18n/README.es.md
@@ -8,10 +8,11 @@
- Construye herramientas empresariales personalizadas en cuestión de minutos y en su propia infraestructura.
+ Construye herramientas empresariales personalizadas en cuestión de minutos y en tu propia infraestructura.
- Budibase es una plataforma de código bajo de código abierto, que ayuda a desarrolladores y profesionales de TI a crear, automatizar y enviar aplicaciones empresariales personalizadas en cuestión de minutos y en su propia infraestructura
+ Budibase es una plataforma low code de código abierto, que ayuda a desarrolladores y profesionales de TI a crear y
+automatizar aplicaciones personalizadas en cuestión de minutos
@@ -20,7 +21,7 @@
- 
+ 
@@ -30,9 +31,6 @@
-
- 
-
@@ -43,130 +41,213 @@
+
+## ✨ Caracteristicas
-## ✨ Features
-When other platforms chose the closed source route, we decided to go open source. When other platforms chose cloud builders, we decided a local builder offered the better developer experience. We like to do things differently at Budibase.
+### Construir aplicaciones reales
+Con Budibase podras construir aplicaciones de pagina unica de gran rendimiento. Ademas, puedes hacerlas con un diseño
+adaptativo para darles a tus usuarios una gran experiencia.
+
-- **Build and ship real software.** Unlike other platforms, with Budibase you build and ship single page applications. Budibase applications have performance baked in and can be designed responsively, providing your users with a great experience.
+### Codigo abierto y ampliable
+Budibase es de codigo abierto con licencia GPL v3. Puedes ampliarlo o modificarlo para adaptarlo a tus necesidades y preferencias.
-- **Open source and extensable.** Budibase is open-source. The builder is licensed AGPL v3, the server is GPL v3, and the client is MPL. This should fill you with confidence that Budibase will always be around. You can also code against Budibase or fork it and make changes as you please, providing a developer-friendly experience.
+De esta manera proveemos una buena experiencia para el desarrollador asi como establecemos la confianza de que Budibase siempre estara funcional.
+
-- **Load data or start from scratch.** Budibase pulls in data from multiple sources, including MongoDB, CouchDB, PostgreSQL, mySQL, Airtable, Google Sheets, S3, DyanmoDB, or a REST API. And unlike other platforms, with Budibase you can start from scratch and create business apps with no data sources. [Request new data sources](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
+### Cargar informacion o empezar desde cero
+Budibase permite importar datos desde multiples fuentes, entre las que estan incluidas: MondoDB, CouchDB, PostgreSQL, MySQL,
+Airtable, S3, DynamoDB o API REST.
-- **Design and build apps with powerful pre-made components.** Budibase comes out of the box with beautifully designed, powerful components which you can use like building blocks to build your UI. We also expose a lot of your favourite CSS styling options so you can go that extra creative mile. [Request new components](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
-
-- **Automate processes, integrate with other tools, and connect to webhooks.** Save time by automating manual processes and workflows. From connecting to webhooks, to automating emails, simply tell Budibase what to do and let it work for you. You can easily [create new automations for Budibase here](https://github.com/Budibase/automations) or [request new integrations here](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
-
-- **Cloud hosting and self-hosting.** Users can self-host (see below), or host their apps with Budibase. Currently, our cloud hosting offering is limited to the free tier but we aim to change this in the future. For heavy usage, we advise users to self-host.
+O si lo prefieres, con Budibase puedes empezar desde cero y construir tus propias aplicaciones
+sin necesidad de herramientas externas.
+[Sugerir fuente de datos](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
- 
+ 
+
+### Diseña y construye aplicaciones con componentes profesionales prediseñados
-## ⌛ Status
-- [x] Alpha: We are demoing Budibase to users and receiving feedback
-- [x] Private Beta: We are testing Budibase with a closed set of customers
-- [x] Public Beta: Anyone can [sign-up and use Budibase](https://portal.budi.live/signup).
-- [ ] Official Launch
+Budibase incorpora componentes profesionales prediseñados que podras usar de manera facil e intuitiva
+como bloques de construccion para la interfaz de tu aplicacion.
-Watch "releases" of this repo to get notified of major updates, and give the star button a click whilst you're there.
+Tambien mostramos gran parte del CSS para que puedas adaptar los componentes a tus diseños.
+[Sugerir componente](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
- 
+ 
+
-### Stargazers over time
+### Procesos automatizados, integra tu aplicacion con otras herramientas y conectala a eventos webhook
+
+Ahorra tiempo automatizando flujos de trabajo y procesos manuales. Podras desde conectar eventos webhook hasta automatizar emails,
+simplemente dile a Budibase que hacer y deja que el haga el trabajo por ti.
+[Crear nuevos procesos automatizados](https://github.com/Budibase/automations) o [Sugerir proceso automatizado](https://github.com/Budibase/budibase/discussions?discussions_q=category%3AIdeas).
+
+
+ 
+
+
+
+### Tus herramientas favoritas
+
+Budibase integra un gran numero de herramientas que te permitiran construir tus aplicaciones ajustandose a tus preferencias.
+
+
+ 
+
+
+
+### Un paraiso para administradores
+
+Puedes albergar Budibase en tu propia infraestructura y gestionar globalmente usuarios, incorporaciones, SMTP, aplicaciones,
+grupos, diseños de temas, etc.
+
+Tambien puedes gestionar los usuarios y grupos, o delegar en personas asignadas para ello, desde nuestra aplicacion sin
+mucho esfuerzo.
+
+Budibase is made to scale. With Budibase, you can self-host on your own infrastructure and globally manage users, onboarding, SMTP, apps, groups, theming and more. You can also provide users/groups with an app portal and disseminate user-management to the group manager.
+
+- Video Promocional: https://youtu.be/xoljVpty_Kw
+
+
+
+---
+
+
+
+
+## Budibase API Publica
+
+Como todo lo que construimos en Budibase, nuestra nueva API publica es facil de usar, flexible e introduce nueva ampliacion
+del sistema. Budibase API ofrece:
+- Uso de Budibase como backend
+- Interoperabilidad
+
+#### Documentacion
+
+Puedes aprender mas acerca de Budibase API en los siguientes documentos:
+- [Documentacion general](https://docs.budibase.com/docs/public-api) : Como optener tu clave para la API, usar Insomnia y Postman
+- [API Interactiva](https://docs.budibase.com/reference/post_applications) : Aprende como trabajar con la API
+
+#### Guias
+
+- [Construye una aplicacion con Budibase y Next.js](https://budibase.com/blog/building-a-crud-app-with-budibase-and-next.js/)
+
+
+ 
+
+
+
+
+
+## 🏁 Comenzar con Budibase
+
+Puedes alojar Budibase en tu propia infraestructura con Docker, Kubernetes o Digital Ocean; o usa Budibase en la nube si
+quieres empezar a crear tus aplicaciones rapidamente y sin ningun tipo de preocupacion.
+
+### [Comenzar con Budibase self-hosting](https://docs.budibase.com/docs/hosting-methods)
+
+- [Docker - single ARM compatible image](https://docs.budibase.com/docs/docker)
+- [Docker Compose](https://docs.budibase.com/docs/docker-compose)
+- [Kubernetes](https://docs.budibase.com/docs/kubernetes-k8s)
+- [Digital Ocean](https://docs.budibase.com/docs/digitalocean)
+- [Portainer](https://docs.budibase.com/docs/portainer)
+
+
+### [Comenzar con Budibase en la nube](https://budibase.com)
+
+
+
+## 🎓 Aprende a usar Budibase
+
+Aqui tienes la [documentacion de Budibase](https://docs.budibase.com/docs).
+
+
+
+
+
+## 💬 Comunidad
+
+Te invitamos a que te unas a nuestra comunidad de Budibase, alli podras hacer las preguntas que quieras, ayudar a otras
+personas o tener una charla entretenida con otros usuarios de Budibase.
+[Acceder a la comunidad de Budibase](https://github.com/Budibase/budibase/discussions)
+
+
+
+## ❗ Codigo de conducta
+
+Budibase presta especial atencion en acoger a personas de toda diversidad y ofrecer un entorno de respeto mutuo. Asi mismo
+esperamos lo mismo de nuestra comunidad, por favor lee el
+[**Codigo de conducta**](https://github.com/Budibase/budibase/blob/HEAD/.github/CODE_OF_CONDUCT.md).
+
+
+
+
+
+## 🙌 Contribuir en Budibase
+
+Desde comunicar un bug a solventar un error en el codigo, toda contribucion es apreciada y bienvenida. Si estas planeando
+implementar una nueva funcionalidad o un realizar un cambio en la API, por favor crea un [nuevo mensaje aqui](https://github.com/Budibase/budibase/issues),
+de esta manera nos encargaremos que tu trabajo no sea en vano.
+
+Aqui tienes instrucciones de como configurar tu entorno Budibase para [Debian](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-DEBIAN.md)
+y [MacOSX](https://github.com/Budibase/budibase/tree/HEAD/docs/DEV-SETUP-MACOSX.md)
+
+### No estas seguro por donde empezar?
+Un buen lugar para empezar a contribuir con nosotros es [aqui](https://github.com/Budibase/budibase/projects/22).
+
+### Organizacion del repositorio
+
+Budibase es un repositorio unico gestionado por Lerna. Lerna construye y publica los paquetes de Budibase sincronizandolos
+cada ves que se realiza un cambio. A rasgos generales, estos son los paquetes que conforman Budibase:
+
+- [packages/builder](https://github.com/Budibase/budibase/tree/HEAD/packages/builder) - contiene el codigo del builder de la parte cliente, esta es una aplicacion svelte.
+
+- [packages/client](https://github.com/Budibase/budibase/tree/HEAD/packages/client) - Este modulo se ejecuta en el browser y es el responsable de leer definiciones JSON y crear aplicaciones web en el momento.
+
+- [packages/server](https://github.com/Budibase/budibase/tree/HEAD/packages/server) - La parte servidor de Budibase. Esta aplicacion Koa es responsable de suministrar lo necesario al builder para asi generar las aplicaciones Budibase. Tambien provee una API para interaccionar con la base de datos y el almacenamiento de ficheros.
+
+Para mas informacion, por favor lee el siguiente documento [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/docs/CONTRIBUTING.md)
+
+
+
+
+## 📝 Licencia
+
+Budibase es open-source, licenciado como [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html). El cliente y las librerias
+de componentes estan licenciadas como [MPL](https://directory.fsf.org/wiki/License:MPL-2.0) - de esta manera, puedes licenciar
+como tu quieras las aplicaciones que construyas.
+
+
+
+## ⭐ Historia de nuestros Stargazers
[](https://starchart.cc/Budibase/budibase)
-If you are having issues between updates of the builder, please use the guide [here](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md#troubleshooting) to clear down your environment.
+Si estas teniendo problemas con el builder despues de actualizar, por favor [lee esta guia](https://github.com/Budibase/budibase/blob/HEAD/docs/CONTRIBUTING.md#troubleshooting) to clear down your environment.
+
-## 🏁 Getting Started with Budibase
+## Contribuidores ✨
-The Budibase builder runs in Electron, on Mac, PC and Linux. Follow the steps below to get started:
-- [ ] [Sign-up to Budibase](https://portal.budi.live/signup)
-- [ ] Create a username and password
-- [ ] Copy your API key
-- [ ] Download Budibase
-- [ ] Open Budibase and enter your API key
-
-[Here is a guided tutorial](https://docs.budibase.com/tutorial/tutorial-signing-up) if you need extra help.
-
-
-## 🤖 Self-hosting
-
-Budibase wants to make sure anyone can use the tools we develop and we know a lot of people need to be able to host the apps they make on their own systems - that is why we've decided to try and make self hosting as easy as possible!
-
-Currently, you can host your apps using Docker or Digital Ocean. The documentation for self-hosting can be found [here](https://docs.budibase.com/docs/hosting-methods).
-
-[](https://cloud.digitalocean.com/droplets/new?onboarding_origin=marketplace&i=09038e&fleetUuid=bb04f9c8-1de8-4687-b2ae-1d5177a0535b&appId=77729671&type=applications&size=s-4vcpu-8gb®ion=nyc1&refcode=0caaa6085a82&image=budibase-20-04)
-
-
-## 🎓 Learning Budibase
-
-The Budibase [documentation lives here](https://docs.budibase.com).
-
-You can also follow a quick tutorial on [how to build a CRM with Budibase](https://docs.budibase.com/tutorial/tutorial-introduction)
-
-
-## Roadmap
-
-Checkout our [Public Roadmap](https://github.com/Budibase/budibase/projects/10). If you would like to discuss some of the items on the roadmap, please feel to reach out on [Discord](https://discord.gg/rCYayfe), or via [Github discussions](https://github.com/Budibase/budibase/discussions)
-
-
-## ❗ Code of Conduct
-
-Budibase is dedicated to providing a welcoming, diverse, and harrassment-free experience for everyone. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/.github/CODE_OF_CONDUCT.md). Please read it.
-
-## 🙌 Contributing to Budibase
-
-From opening a bug report to creating a pull request: every contribution is appreciated and welcomed. If you're planning to implement a new feature or change the API please create an issue first. This way we can ensure your work is not in vain.
-
-### Not Sure Where to Start?
-A good place to start contributing, is the [First time issues project](https://github.com/Budibase/budibase/projects/22).
-
-### How the repository is organized
-Budibase is a monorepo managed by lerna. Lerna manages the building and publishing of the budibase packages. At a high level, here are the packages that make up Budibase.
-
-- [packages/builder](https://github.com/Budibase/budibase/tree/HEAD/packages/builder) - contains code for the budibase builder client side svelte application.
-
-- [packages/client](https://github.com/Budibase/budibase/tree/HEAD/packages/client) - A module that runs in the browser responsible for reading JSON definition and creating living, breathing web apps from it.
-
-- [packages/server](https://github.com/Budibase/budibase/tree/HEAD/packages/server) - The budibase server. This Koa app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system.
-
-For more information, see [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md)
-
-## 📝 License
-
-Budibase is open-source. The builder is licensed [AGPL v3](https://www.gnu.org/licenses/agpl-3.0.en.html), the server is licensed [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html), and the client is licensed [MPL](https://directory.fsf.org/wiki/License:MPL-2.0).
-
-## 💬 Get in touch
-
-If you have a question or would like to talk with other Budibase users, please hop over to [Github discussions](https://github.com/Budibase/budibase/discussions) or join our Discord server:
-
-[Discord chatroom](https://discord.gg/rCYayfe)
-
-
-
-
-## Contributors ✨
-
-Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
+Queremos prestar un especial agradecimiento a nuestra maravillosa gente ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
@@ -179,14 +260,18 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
Michael Shanks
📖 💻 ⚠️ |
Kevin Åberg Kultalahti
📖 💻 ⚠️ |
Joe
📖 💻 🖋 🎨 |
- 
Conor_Mack
💻 ⚠️ |
+ 
Rory Powell
💻 📖 ⚠️ |
+ 
Peter Clement
💻 📖 ⚠️ |
+
Conor_Mack
💻 ⚠️ |
pngwn
💻 ⚠️ |
HugoLd
💻 |
victoriasloan
💻 |
yashank09
💻 |
SOVLOOKUP
💻 |
+ 
seoulaja
🌍 |
+ 
Maurits Lourens
⚠️ 💻 |
@@ -195,4 +280,5 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
-This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
+Este proyecto sigue las especificaciones de [all-contributors](https://github.com/all-contributors/all-contributors).
+Todo tipo de contribuciones son agradecidas!
diff --git a/lerna.json b/lerna.json
index 9571899f44..8ce0e0c314 100644
--- a/lerna.json
+++ b/lerna.json
@@ -1,5 +1,5 @@
{
- "version": "1.2.5",
+ "version": "1.2.14",
"npmClient": "yarn",
"packages": [
"packages/*"
diff --git a/packages/backend-core/package.json b/packages/backend-core/package.json
index 1515c4b4b6..6081d5b8e6 100644
--- a/packages/backend-core/package.json
+++ b/packages/backend-core/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/backend-core",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase backend core libraries used in server and worker",
"main": "dist/src/index.js",
"types": "dist/src/index.d.ts",
@@ -20,7 +20,7 @@
"test:watch": "jest --watchAll"
},
"dependencies": {
- "@budibase/types": "^1.2.5",
+ "@budibase/types": "^1.2.14",
"@techpass/passport-openidconnect": "0.3.2",
"aws-sdk": "2.1030.0",
"bcrypt": "5.0.1",
diff --git a/packages/backend-core/src/auth.js b/packages/backend-core/src/auth.js
index 9ae29a3cbd..d39b8426fb 100644
--- a/packages/backend-core/src/auth.js
+++ b/packages/backend-core/src/auth.js
@@ -19,6 +19,8 @@ const {
csrf,
internalApi,
adminOnly,
+ builderOnly,
+ builderOrAdmin,
joiValidator,
} = require("./middleware")
@@ -176,5 +178,7 @@ module.exports = {
updateUserOAuth,
ssoCallbackUrl,
adminOnly,
+ builderOnly,
+ builderOrAdmin,
joiValidator,
}
diff --git a/packages/backend-core/src/environment.ts b/packages/backend-core/src/environment.ts
index 37804b31a6..51cc721ded 100644
--- a/packages/backend-core/src/environment.ts
+++ b/packages/backend-core/src/environment.ts
@@ -55,6 +55,7 @@ const env = {
DEFAULT_LICENSE: process.env.DEFAULT_LICENSE,
SERVICE: process.env.SERVICE || "budibase",
MEMORY_LEAK_CHECK: process.env.MEMORY_LEAK_CHECK || false,
+ LOG_LEVEL: process.env.LOG_LEVEL,
DEPLOYMENT_ENVIRONMENT:
process.env.DEPLOYMENT_ENVIRONMENT || "docker-compose",
_set(key: any, value: any) {
diff --git a/packages/backend-core/src/index.ts b/packages/backend-core/src/index.ts
index 35777ae817..ced4630fb7 100644
--- a/packages/backend-core/src/index.ts
+++ b/packages/backend-core/src/index.ts
@@ -15,6 +15,7 @@ import auth from "./auth"
import constants from "./constants"
import * as dbConstants from "./db/constants"
import logging from "./logging"
+import pino from "./pino"
// mimic the outer package exports
import * as db from "./pkg/db"
@@ -53,6 +54,7 @@ const core = {
errors,
logging,
roles,
+ ...pino,
...errorClasses,
}
diff --git a/packages/backend-core/src/middleware/authenticated.js b/packages/backend-core/src/middleware/authenticated.js
index d86af773c3..674c16aa55 100644
--- a/packages/backend-core/src/middleware/authenticated.js
+++ b/packages/backend-core/src/middleware/authenticated.js
@@ -81,7 +81,7 @@ module.exports = (
const session = await getSession(userId, sessionId)
if (!session) {
- error = "No session found"
+ error = `Session not found - ${userId} - ${sessionId}`
} else {
try {
if (opts && opts.populateUser) {
diff --git a/packages/worker/src/middleware/builderOnly.js b/packages/backend-core/src/middleware/builderOnly.js
similarity index 100%
rename from packages/worker/src/middleware/builderOnly.js
rename to packages/backend-core/src/middleware/builderOnly.js
diff --git a/packages/worker/src/middleware/builderOrAdmin.js b/packages/backend-core/src/middleware/builderOrAdmin.js
similarity index 100%
rename from packages/worker/src/middleware/builderOrAdmin.js
rename to packages/backend-core/src/middleware/builderOrAdmin.js
diff --git a/packages/backend-core/src/middleware/index.js b/packages/backend-core/src/middleware/index.js
index 9d94bf5763..7e7b8a2931 100644
--- a/packages/backend-core/src/middleware/index.js
+++ b/packages/backend-core/src/middleware/index.js
@@ -10,6 +10,8 @@ const internalApi = require("./internalApi")
const datasourceGoogle = require("./passport/datasource/google")
const csrf = require("./csrf")
const adminOnly = require("./adminOnly")
+const builderOrAdmin = require("./builderOrAdmin")
+const builderOnly = require("./builderOnly")
const joiValidator = require("./joi-validator")
module.exports = {
google,
@@ -27,5 +29,7 @@ module.exports = {
},
csrf,
adminOnly,
+ builderOnly,
+ builderOrAdmin,
joiValidator,
}
diff --git a/packages/backend-core/src/migrations/definitions.ts b/packages/backend-core/src/migrations/definitions.ts
index 745c8718c9..34ec0f0cad 100644
--- a/packages/backend-core/src/migrations/definitions.ts
+++ b/packages/backend-core/src/migrations/definitions.ts
@@ -37,4 +37,8 @@ export const DEFINITIONS: MigrationDefinition[] = [
type: MigrationType.INSTALLATION,
name: MigrationName.EVENT_INSTALLATION_BACKFILL,
},
+ {
+ type: MigrationType.GLOBAL,
+ name: MigrationName.GLOBAL_INFO_SYNC_USERS,
+ },
]
diff --git a/packages/backend-core/src/pino.js b/packages/backend-core/src/pino.js
new file mode 100644
index 0000000000..69962b3841
--- /dev/null
+++ b/packages/backend-core/src/pino.js
@@ -0,0 +1,11 @@
+const env = require("./environment")
+
+exports.pinoSettings = () => ({
+ prettyPrint: {
+ levelFirst: true,
+ },
+ level: env.LOG_LEVEL || "error",
+ autoLogging: {
+ ignore: req => req.url.includes("/health"),
+ },
+})
diff --git a/packages/backend-core/src/security/sessions.js b/packages/backend-core/src/security/sessions.js
index 8874b47469..a3be0a1a58 100644
--- a/packages/backend-core/src/security/sessions.js
+++ b/packages/backend-core/src/security/sessions.js
@@ -1,5 +1,7 @@
const redis = require("../redis/init")
const { v4: uuidv4 } = require("uuid")
+const { logWarn } = require("../logging")
+const env = require("../environment")
// a week in seconds
const EXPIRY_SECONDS = 86400 * 7
@@ -33,12 +35,21 @@ async function invalidateSessions(userId, sessionIds = null) {
}))
}
- const client = await redis.getSessionClient()
- const promises = []
- for (let session of sessions) {
- promises.push(client.delete(session.key))
+ if (sessions && sessions.length > 0) {
+ const client = await redis.getSessionClient()
+ const promises = []
+ for (let session of sessions) {
+ promises.push(client.delete(session.key))
+ }
+ if (!env.isTest()) {
+ logWarn(
+ `Invalidating sessions for ${userId} - ${sessions
+ .map(session => session.key)
+ .join(", ")}`
+ )
+ }
+ await Promise.all(promises)
}
- await Promise.all(promises)
} catch (err) {
console.error(`Error invalidating sessions: ${err}`)
}
diff --git a/packages/bbui/package.json b/packages/bbui/package.json
index c91176a8b7..342e65583d 100644
--- a/packages/bbui/package.json
+++ b/packages/bbui/package.json
@@ -1,7 +1,7 @@
{
"name": "@budibase/bbui",
"description": "A UI solution used in the different Budibase projects.",
- "version": "1.2.5",
+ "version": "1.2.14",
"license": "MPL-2.0",
"svelte": "src/index.js",
"module": "dist/bbui.es.js",
@@ -38,7 +38,7 @@
],
"dependencies": {
"@adobe/spectrum-css-workflow-icons": "^1.2.1",
- "@budibase/string-templates": "^1.2.5",
+ "@budibase/string-templates": "^1.2.14",
"@spectrum-css/actionbutton": "^1.0.1",
"@spectrum-css/actiongroup": "^1.0.1",
"@spectrum-css/avatar": "^3.0.2",
diff --git a/packages/builder/package.json b/packages/builder/package.json
index aff7d5f8a6..a5f1e5ecd6 100644
--- a/packages/builder/package.json
+++ b/packages/builder/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/builder",
- "version": "1.2.5",
+ "version": "1.2.14",
"license": "GPL-3.0",
"private": true,
"scripts": {
@@ -69,10 +69,10 @@
}
},
"dependencies": {
- "@budibase/bbui": "^1.2.5",
- "@budibase/client": "^1.2.5",
- "@budibase/frontend-core": "^1.2.5",
- "@budibase/string-templates": "^1.2.5",
+ "@budibase/bbui": "^1.2.14",
+ "@budibase/client": "^1.2.14",
+ "@budibase/frontend-core": "^1.2.14",
+ "@budibase/string-templates": "^1.2.14",
"@sentry/browser": "5.19.1",
"@spectrum-css/page": "^3.0.1",
"@spectrum-css/vars": "^3.0.1",
diff --git a/packages/cli/.gitignore b/packages/cli/.gitignore
index efef4f97c8..655ef7b624 100644
--- a/packages/cli/.gitignore
+++ b/packages/cli/.gitignore
@@ -5,3 +5,4 @@ build/
docker-error.log
envoy.yaml
*.tar.gz
+prebuilds/
diff --git a/packages/cli/package.json b/packages/cli/package.json
index 5aa0a12a52..b94d8ef4fb 100644
--- a/packages/cli/package.json
+++ b/packages/cli/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/cli",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase CLI, for developers, self hosting and migrations.",
"main": "src/index.js",
"bin": {
diff --git a/packages/client/package.json b/packages/client/package.json
index 614849108a..4e2daac226 100644
--- a/packages/client/package.json
+++ b/packages/client/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/client",
- "version": "1.2.5",
+ "version": "1.2.14",
"license": "MPL-2.0",
"module": "dist/budibase-client.js",
"main": "dist/budibase-client.js",
@@ -19,9 +19,9 @@
"dev:builder": "rollup -cw"
},
"dependencies": {
- "@budibase/bbui": "^1.2.5",
- "@budibase/frontend-core": "^1.2.5",
- "@budibase/string-templates": "^1.2.5",
+ "@budibase/bbui": "^1.2.14",
+ "@budibase/frontend-core": "^1.2.14",
+ "@budibase/string-templates": "^1.2.14",
"@spectrum-css/button": "^3.0.3",
"@spectrum-css/card": "^3.0.3",
"@spectrum-css/divider": "^1.0.3",
diff --git a/packages/frontend-core/package.json b/packages/frontend-core/package.json
index 8dbde0b8c8..3d8e52cf74 100644
--- a/packages/frontend-core/package.json
+++ b/packages/frontend-core/package.json
@@ -1,12 +1,12 @@
{
"name": "@budibase/frontend-core",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase frontend core libraries used in builder and client",
"author": "Budibase",
"license": "MPL-2.0",
"svelte": "src/index.js",
"dependencies": {
- "@budibase/bbui": "^1.2.5",
+ "@budibase/bbui": "^1.2.14",
"lodash": "^4.17.21",
"svelte": "^3.46.2"
}
diff --git a/packages/server/package.json b/packages/server/package.json
index 761247d755..9052ea132e 100644
--- a/packages/server/package.json
+++ b/packages/server/package.json
@@ -1,7 +1,7 @@
{
"name": "@budibase/server",
"email": "hi@budibase.com",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase Web Server",
"main": "src/index.ts",
"repository": {
@@ -77,11 +77,11 @@
"license": "GPL-3.0",
"dependencies": {
"@apidevtools/swagger-parser": "10.0.3",
- "@budibase/backend-core": "^1.2.5",
- "@budibase/client": "^1.2.5",
- "@budibase/pro": "1.2.5",
- "@budibase/string-templates": "^1.2.5",
- "@budibase/types": "^1.2.5",
+ "@budibase/backend-core": "^1.2.14",
+ "@budibase/client": "^1.2.14",
+ "@budibase/pro": "1.2.14",
+ "@budibase/string-templates": "^1.2.14",
+ "@budibase/types": "^1.2.14",
"@bull-board/api": "3.7.0",
"@bull-board/koa": "3.9.4",
"@elastic/elasticsearch": "7.10.0",
diff --git a/packages/server/src/app.ts b/packages/server/src/app.ts
index 32951cc47e..62301d57ca 100644
--- a/packages/server/src/app.ts
+++ b/packages/server/src/app.ts
@@ -15,6 +15,7 @@ const Sentry = require("@sentry/node")
const fileSystem = require("./utilities/fileSystem")
const bullboard = require("./automations/bullboard")
const { logAlert } = require("@budibase/backend-core/logging")
+const { pinoSettings } = require("@budibase/backend-core")
const { Thread } = require("./threads")
import redis from "./utilities/redis"
import * as migrations from "./migrations"
@@ -35,14 +36,7 @@ app.use(
})
)
-app.use(
- pino({
- prettyPrint: {
- levelFirst: true,
- },
- level: env.LOG_LEVEL || "error",
- })
-)
+app.use(pino(pinoSettings()))
if (!env.isTest()) {
const plugin = bullboard.init()
diff --git a/packages/server/yarn.lock b/packages/server/yarn.lock
index d957ef6812..2d7a376ab1 100644
--- a/packages/server/yarn.lock
+++ b/packages/server/yarn.lock
@@ -1094,12 +1094,12 @@
resolved "https://registry.yarnpkg.com/@bcoe/v8-coverage/-/v8-coverage-0.2.3.tgz#75a2e8b51cb758a7553d6804a5932d7aace75c39"
integrity sha512-0hYQ8SB4Db5zvZB4axdMHGwEaQjkZzFjQiN9LVYvIFB2nSUHW9tYpxWriPrWDASIxiaXax83REcLxuSdnGPZtw==
-"@budibase/backend-core@1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-1.2.5.tgz#4d8e9e86587f73244c298c4483715b5754e1eebd"
- integrity sha512-8HChhwa6Mo4FouNAWtai5F9FugGCg5EQ8nn47/W5O+WM/VaWdJcjGURHfR9KnWHQUOyWlonvhHEgzyFWp5t+7w==
+"@budibase/backend-core@1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-1.2.14.tgz#1aca6a82b8ac28f5c101721ba4596f674eb57197"
+ integrity sha512-870ZzNX13/f1U7qkUJUoKp/e8UQqDHEc0TZBBaWcpLXbEBs6ikxGp5YdYGEVXkr/ca4rusfHs6jAjUNkIPThow==
dependencies:
- "@budibase/types" "^1.2.5"
+ "@budibase/types" "^1.2.14"
"@techpass/passport-openidconnect" "0.3.2"
aws-sdk "2.1030.0"
bcrypt "5.0.1"
@@ -1177,13 +1177,13 @@
svelte-flatpickr "^3.2.3"
svelte-portal "^1.0.0"
-"@budibase/pro@1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-1.2.5.tgz#3a0c2adc994027e114bb133ac26f0562bdb7c945"
- integrity sha512-//RB2p5FVkhH8ylMFyaU0Z1Gh4d6x9p4lm5wabe282O1sIdBAbsq9mtMK6NIftZgakELHS/xwOR6W/knKB4Ybw==
+"@budibase/pro@1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-1.2.14.tgz#bce72c8a0d48dbec8dc6aa4af47abb9dbfd3d029"
+ integrity sha512-nYyVHO1IaoDRFSrLp0ueKWG0/Z+PSRs3yF+OZ7vQaECCXXSgWnZyCOy9o6iIovUlbimbPxrQvkwcL8v4deWuvQ==
dependencies:
- "@budibase/backend-core" "1.2.5"
- "@budibase/types" "1.2.5"
+ "@budibase/backend-core" "1.2.14"
+ "@budibase/types" "1.2.14"
"@koa/router" "8.0.8"
joi "17.6.0"
node-fetch "^2.6.1"
@@ -1206,10 +1206,10 @@
svelte-apexcharts "^1.0.2"
svelte-flatpickr "^3.1.0"
-"@budibase/types@1.2.5", "@budibase/types@^1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/types/-/types-1.2.5.tgz#0d4dafdee42fb5324dba9393206c6e34d4538afe"
- integrity sha512-ZEDy3TlVSWEhQy7HkAqsuZWJmFxuELo6pu77mpH2fEHY+UpGkSSQPxMg7LYYuWDNsGjJGl0urXh0G5StDM1aTw==
+"@budibase/types@1.2.14", "@budibase/types@^1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/types/-/types-1.2.14.tgz#8ef2d1b526661629dd9db2f10fea515fadfdcc53"
+ integrity sha512-E3k1kJz24W0l1aw1cL2Edf7WnO6Nt6iew4stDpuQjG2r1EcXPv6/hPsRA7Ms0331ZjGKEswaddSyF5NTn1O8VQ==
"@bull-board/api@3.7.0":
version "3.7.0"
diff --git a/packages/string-templates/package.json b/packages/string-templates/package.json
index 10f3d13e06..5d859902b5 100644
--- a/packages/string-templates/package.json
+++ b/packages/string-templates/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/string-templates",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Handlebars wrapper for Budibase templating.",
"main": "src/index.cjs",
"module": "dist/bundle.mjs",
diff --git a/packages/types/package.json b/packages/types/package.json
index 55530259e9..c62ed636d1 100644
--- a/packages/types/package.json
+++ b/packages/types/package.json
@@ -1,6 +1,6 @@
{
"name": "@budibase/types",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase types",
"main": "dist/index.js",
"types": "dist/index.d.ts",
diff --git a/packages/types/src/sdk/migrations.ts b/packages/types/src/sdk/migrations.ts
index bb32d2e045..23a4d6d097 100644
--- a/packages/types/src/sdk/migrations.ts
+++ b/packages/types/src/sdk/migrations.ts
@@ -46,6 +46,7 @@ export enum MigrationName {
EVENT_APP_BACKFILL = "event_app_backfill",
EVENT_GLOBAL_BACKFILL = "event_global_backfill",
EVENT_INSTALLATION_BACKFILL = "event_installation_backfill",
+ GLOBAL_INFO_SYNC_USERS = "global_info_sync_users",
}
export interface MigrationDefinition {
diff --git a/packages/worker/package.json b/packages/worker/package.json
index 000ad623c8..70afd899e8 100644
--- a/packages/worker/package.json
+++ b/packages/worker/package.json
@@ -1,7 +1,7 @@
{
"name": "@budibase/worker",
"email": "hi@budibase.com",
- "version": "1.2.5",
+ "version": "1.2.14",
"description": "Budibase background service",
"main": "src/index.ts",
"repository": {
@@ -35,10 +35,10 @@
"author": "Budibase",
"license": "GPL-3.0",
"dependencies": {
- "@budibase/backend-core": "^1.2.5",
- "@budibase/pro": "1.2.5",
- "@budibase/string-templates": "^1.2.5",
- "@budibase/types": "^1.2.5",
+ "@budibase/backend-core": "^1.2.14",
+ "@budibase/pro": "1.2.14",
+ "@budibase/string-templates": "^1.2.14",
+ "@budibase/types": "^1.2.14",
"@koa/router": "8.0.8",
"@sentry/node": "6.17.7",
"@techpass/passport-openidconnect": "0.3.2",
diff --git a/packages/worker/src/api/controllers/global/sessions.js b/packages/worker/src/api/controllers/global/sessions.js
deleted file mode 100644
index 4a334037d4..0000000000
--- a/packages/worker/src/api/controllers/global/sessions.js
+++ /dev/null
@@ -1,37 +0,0 @@
-const {
- getAllSessions,
- getUserSessions,
- invalidateSessions,
-} = require("@budibase/backend-core/sessions")
-
-exports.fetch = async ctx => {
- ctx.body = await getAllSessions()
-}
-
-exports.find = async ctx => {
- const { userId } = ctx.params
- const sessions = await getUserSessions(userId)
- ctx.body = sessions.map(session => session.value)
-}
-
-exports.invalidateUser = async ctx => {
- const { userId } = ctx.params
- await invalidateSessions(userId)
- ctx.body = {
- message: "User sessions invalidated",
- }
-}
-
-exports.selfSessions = async ctx => {
- const userId = ctx.user._id
- ctx.body = await getUserSessions(userId)
-}
-
-exports.invalidateSession = async ctx => {
- const userId = ctx.user._id
- const { sessionId } = ctx.params
- await invalidateSessions(userId, sessionId)
- ctx.body = {
- message: "Session invalidated successfully.",
- }
-}
diff --git a/packages/worker/src/api/controllers/system/migrations.ts b/packages/worker/src/api/controllers/system/migrations.ts
new file mode 100644
index 0000000000..57a5f6261c
--- /dev/null
+++ b/packages/worker/src/api/controllers/system/migrations.ts
@@ -0,0 +1,13 @@
+const { migrate, MIGRATIONS } = require("../../../migrations")
+
+export const runMigrations = async (ctx: any) => {
+ const options = ctx.request.body
+ // don't await as can take a while, just return
+ migrate(options)
+ ctx.status = 200
+}
+
+export const fetchDefinitions = async (ctx: any) => {
+ ctx.body = MIGRATIONS
+ ctx.status = 200
+}
diff --git a/packages/worker/src/api/index.js b/packages/worker/src/api/index.js
index 281d9d097c..ca56e0c5d2 100644
--- a/packages/worker/src/api/index.js
+++ b/packages/worker/src/api/index.js
@@ -106,7 +106,10 @@ router
if (ctx.publicEndpoint) {
return next()
}
- if ((!ctx.isAuthenticated || !ctx.user.budibaseAccess) && !ctx.internal) {
+ if (
+ (!ctx.isAuthenticated || (ctx.user && !ctx.user.budibaseAccess)) &&
+ !ctx.internal
+ ) {
ctx.throw(403, "Unauthorized - no public worker access")
}
return next()
diff --git a/packages/worker/src/api/routes/global/roles.js b/packages/worker/src/api/routes/global/roles.js
index 4e27b7d54b..d99e0e5b56 100644
--- a/packages/worker/src/api/routes/global/roles.js
+++ b/packages/worker/src/api/routes/global/roles.js
@@ -1,12 +1,12 @@
const Router = require("@koa/router")
const controller = require("../../controllers/global/roles")
-const { adminOnly } = require("@budibase/backend-core/auth")
+const { builderOrAdmin } = require("@budibase/backend-core/auth")
const router = Router()
router
- .get("/api/global/roles", adminOnly, controller.fetch)
- .get("/api/global/roles/:appId", adminOnly, controller.find)
- .delete("/api/global/roles/:appId", adminOnly, controller.removeAppRole)
+ .get("/api/global/roles", builderOrAdmin, controller.fetch)
+ .get("/api/global/roles/:appId", builderOrAdmin, controller.find)
+ .delete("/api/global/roles/:appId", builderOrAdmin, controller.removeAppRole)
module.exports = router
diff --git a/packages/worker/src/api/routes/global/self.js b/packages/worker/src/api/routes/global/self.js
index e1af7c2146..1683a94f37 100644
--- a/packages/worker/src/api/routes/global/self.js
+++ b/packages/worker/src/api/routes/global/self.js
@@ -1,6 +1,6 @@
const Router = require("@koa/router")
const controller = require("../../controllers/global/self")
-const builderOnly = require("../../../middleware/builderOnly")
+const { builderOnly } = require("@budibase/backend-core/auth")
const { users } = require("../validation")
const router = Router()
diff --git a/packages/worker/src/api/routes/global/sessions.js b/packages/worker/src/api/routes/global/sessions.js
deleted file mode 100644
index 6ab4ad8e59..0000000000
--- a/packages/worker/src/api/routes/global/sessions.js
+++ /dev/null
@@ -1,14 +0,0 @@
-const Router = require("@koa/router")
-const controller = require("../../controllers/global/sessions")
-const { adminOnly } = require("@budibase/backend-core/auth")
-
-const router = Router()
-
-router
- .get("/api/global/sessions", adminOnly, controller.fetch)
- .get("/api/global/sessions/self", controller.selfSessions)
- .get("/api/global/sessions/:userId", adminOnly, controller.find)
- .delete("/api/global/sessions/:userId", adminOnly, controller.invalidateUser)
- .delete("/api/global/sessions/self/:sessionId", controller.invalidateSession)
-
-module.exports = router
diff --git a/packages/worker/src/api/routes/global/users.js b/packages/worker/src/api/routes/global/users.js
index 0fc479df39..e0a221a795 100644
--- a/packages/worker/src/api/routes/global/users.js
+++ b/packages/worker/src/api/routes/global/users.js
@@ -6,7 +6,7 @@ const Joi = require("joi")
const cloudRestricted = require("../../../middleware/cloudRestricted")
const { users } = require("../validation")
const selfController = require("../../controllers/global/self")
-const builderOrAdmin = require("../../../middleware/builderOrAdmin")
+const { builderOrAdmin } = require("@budibase/backend-core/auth")
const router = Router()
diff --git a/packages/worker/src/api/routes/index.js b/packages/worker/src/api/routes/index.js
index 89c67bdf88..550d14a9a3 100644
--- a/packages/worker/src/api/routes/index.js
+++ b/packages/worker/src/api/routes/index.js
@@ -6,12 +6,12 @@ const templateRoutes = require("./global/templates")
const emailRoutes = require("./global/email")
const authRoutes = require("./global/auth")
const roleRoutes = require("./global/roles")
-const sessionRoutes = require("./global/sessions")
const environmentRoutes = require("./system/environment")
const tenantsRoutes = require("./system/tenants")
const statusRoutes = require("./system/status")
const selfRoutes = require("./global/self")
const licenseRoutes = require("./global/license")
+const migrationRoutes = require("./system/migrations")
let userGroupRoutes = api.groups
exports.routes = [
@@ -22,11 +22,11 @@ exports.routes = [
templateRoutes,
tenantsRoutes,
emailRoutes,
- sessionRoutes,
roleRoutes,
environmentRoutes,
statusRoutes,
selfRoutes,
licenseRoutes,
userGroupRoutes,
+ migrationRoutes,
]
diff --git a/packages/worker/src/api/routes/system/migrations.ts b/packages/worker/src/api/routes/system/migrations.ts
new file mode 100644
index 0000000000..5dcf90c4de
--- /dev/null
+++ b/packages/worker/src/api/routes/system/migrations.ts
@@ -0,0 +1,19 @@
+import Router from "@koa/router"
+import * as migrationsController from "../../controllers/system/migrations"
+import { auth } from "@budibase/backend-core"
+
+const router = new Router()
+
+router
+ .post(
+ "/api/system/migrations/run",
+ auth.internalApi,
+ migrationsController.runMigrations
+ )
+ .get(
+ "/api/system/migrations/definitions",
+ auth.internalApi,
+ migrationsController.fetchDefinitions
+ )
+
+export = router
diff --git a/packages/worker/src/api/routes/validation/users.ts b/packages/worker/src/api/routes/validation/users.ts
index e7ad4cca18..d84ae94ee6 100644
--- a/packages/worker/src/api/routes/validation/users.ts
+++ b/packages/worker/src/api/routes/validation/users.ts
@@ -1,4 +1,4 @@
-import joiValidator from "../../../middleware/joi-validator"
+const { joiValidator } = require("@budibase/backend-core/auth")
import Joi from "joi"
let schema: any = {
diff --git a/packages/worker/src/index.ts b/packages/worker/src/index.ts
index f8031abacb..6fb954a1b5 100644
--- a/packages/worker/src/index.ts
+++ b/packages/worker/src/index.ts
@@ -18,7 +18,7 @@ const http = require("http")
const api = require("./api")
const redis = require("./utilities/redis")
const Sentry = require("@sentry/node")
-import { events } from "@budibase/backend-core"
+import { events, pinoSettings } from "@budibase/backend-core"
// this will setup http and https proxies form env variables
bootstrap()
@@ -30,14 +30,7 @@ app.keys = ["secret", "key"]
// set up top level koa middleware
app.use(koaBody({ multipart: true }))
app.use(koaSession(app))
-app.use(
- logger({
- prettyPrint: {
- levelFirst: true,
- },
- level: env.LOG_LEVEL || "error",
- })
-)
+app.use(logger(pinoSettings()))
// authentication
app.use(passport.initialize())
diff --git a/packages/worker/src/middleware/adminOnly.js b/packages/worker/src/middleware/adminOnly.js
deleted file mode 100644
index 4bfdf83848..0000000000
--- a/packages/worker/src/middleware/adminOnly.js
+++ /dev/null
@@ -1,9 +0,0 @@
-module.exports = async (ctx, next) => {
- if (
- !ctx.internal &&
- (!ctx.user || !ctx.user.admin || !ctx.user.admin.global)
- ) {
- ctx.throw(403, "Admin user only endpoint.")
- }
- return next()
-}
diff --git a/packages/worker/src/middleware/joi-validator.js b/packages/worker/src/middleware/joi-validator.js
deleted file mode 100644
index 1686b0e727..0000000000
--- a/packages/worker/src/middleware/joi-validator.js
+++ /dev/null
@@ -1,28 +0,0 @@
-function validate(schema, property) {
- // Return a Koa middleware function
- return (ctx, next) => {
- if (!schema) {
- return next()
- }
- let params = null
- if (ctx[property] != null) {
- params = ctx[property]
- } else if (ctx.request[property] != null) {
- params = ctx.request[property]
- }
- const { error } = schema.validate(params)
- if (error) {
- ctx.throw(400, `Invalid ${property} - ${error.message}`)
- return
- }
- return next()
- }
-}
-
-module.exports.body = schema => {
- return validate(schema, "body")
-}
-
-module.exports.params = schema => {
- return validate(schema, "params")
-}
diff --git a/packages/worker/src/migrations/functions/globalInfoSyncUsers.ts b/packages/worker/src/migrations/functions/globalInfoSyncUsers.ts
new file mode 100644
index 0000000000..cae6c6af51
--- /dev/null
+++ b/packages/worker/src/migrations/functions/globalInfoSyncUsers.ts
@@ -0,0 +1,20 @@
+import { User } from "@budibase/types"
+import * as sdk from "../../sdk"
+
+/**
+ * Date:
+ * Aug 2022
+ *
+ * Description:
+ * Re-sync the global-db users to the global-info db users
+ */
+export const run = async (globalDb: any) => {
+ const users = (await sdk.users.allUsers()) as User[]
+ const promises = []
+ for (let user of users) {
+ promises.push(
+ sdk.users.addTenant(user.tenantId, user._id as string, user.email)
+ )
+ }
+ await Promise.all(promises)
+}
diff --git a/packages/worker/src/migrations/index.ts b/packages/worker/src/migrations/index.ts
new file mode 100644
index 0000000000..6900596216
--- /dev/null
+++ b/packages/worker/src/migrations/index.ts
@@ -0,0 +1,74 @@
+import { migrations, redis } from "@budibase/backend-core"
+import { Migration, MigrationOptions, MigrationName } from "@budibase/types"
+import env from "../environment"
+
+// migration functions
+import * as syncUserInfo from "./functions/globalInfoSyncUsers"
+
+/**
+ * Populate the migration function and additional configuration from
+ * the static migration definitions.
+ */
+export const buildMigrations = () => {
+ const definitions = migrations.DEFINITIONS
+ const workerMigrations: Migration[] = []
+
+ for (const definition of definitions) {
+ switch (definition.name) {
+ case MigrationName.GLOBAL_INFO_SYNC_USERS: {
+ // only needed in cloud
+ if (!env.SELF_HOSTED) {
+ workerMigrations.push({
+ ...definition,
+ fn: syncUserInfo.run,
+ })
+ }
+ break
+ }
+ }
+ }
+
+ return workerMigrations
+}
+
+export const MIGRATIONS = buildMigrations()
+
+export const migrate = async (options?: MigrationOptions) => {
+ if (env.SELF_HOSTED) {
+ await migrateWithLock(options)
+ } else {
+ await migrations.runMigrations(MIGRATIONS, options)
+ }
+}
+
+const migrateWithLock = async (options?: MigrationOptions) => {
+ // get a new lock client
+ const redlock = await redis.clients.getMigrationsRedlock()
+ // lock for 15 minutes
+ const ttl = 1000 * 60 * 15
+
+ let migrationLock
+
+ // acquire lock
+ try {
+ migrationLock = await redlock.lock("migrations", ttl)
+ } catch (e: any) {
+ if (e.name === "LockError") {
+ return
+ } else {
+ throw e
+ }
+ }
+
+ // run migrations
+ try {
+ await migrations.runMigrations(MIGRATIONS, options)
+ } finally {
+ // release lock
+ try {
+ await migrationLock.unlock()
+ } catch (e) {
+ console.error("unable to release migration lock")
+ }
+ }
+}
diff --git a/packages/worker/src/sdk/users/users.ts b/packages/worker/src/sdk/users/users.ts
index ea7f2517e0..e6b3f0a21d 100644
--- a/packages/worker/src/sdk/users/users.ts
+++ b/packages/worker/src/sdk/users/users.ts
@@ -101,12 +101,11 @@ interface SaveUserOpts {
bulkCreate?: boolean
}
-export const buildUser = async (
+const buildUser = async (
user: any,
opts: SaveUserOpts = {
hashPassword: true,
requirePassword: true,
- bulkCreate: false,
},
tenantId: string,
dbUser?: any
@@ -185,15 +184,12 @@ export const save = async (
dbUser = await db.get(_id)
}
- let builtUser = await buildUser(
- user,
- {
- hashPassword: true,
- requirePassword: user.requirePassword,
- },
- tenantId,
- dbUser
- )
+ let builtUser = await buildUser(user, opts, tenantId, dbUser)
+
+ // make sure we set the _id field for a new user
+ if (!_id) {
+ _id = builtUser._id
+ }
try {
const putOpts = {
@@ -220,7 +216,7 @@ export const save = async (
await addTenant(tenantId, _id, email)
await cache.user.invalidateUser(response.id)
// let server know to sync user
- await apps.syncUserInApps(builtUser._id)
+ await apps.syncUserInApps(_id)
return {
_id: response.id,
@@ -293,7 +289,6 @@ export const bulkCreate = async (
{
hashPassword: true,
requirePassword: user.requirePassword,
- bulkCreate: false,
},
tenantId
)
@@ -305,6 +300,9 @@ export const bulkCreate = async (
// Post processing of bulk added users, i.e events and cache operations
for (const user of usersToBulkSave) {
+ // TODO: Refactor to bulk insert users into the info db
+ // instead of relying on looping tenant creation
+ await addTenant(tenantId, user._id, user.email)
await eventHelpers.handleSaveEvents(user, null)
await apps.syncUserInApps(user._id)
}
diff --git a/packages/worker/yarn.lock b/packages/worker/yarn.lock
index 7f0700015f..e03cbaf580 100644
--- a/packages/worker/yarn.lock
+++ b/packages/worker/yarn.lock
@@ -291,12 +291,12 @@
resolved "https://registry.yarnpkg.com/@bcoe/v8-coverage/-/v8-coverage-0.2.3.tgz#75a2e8b51cb758a7553d6804a5932d7aace75c39"
integrity sha512-0hYQ8SB4Db5zvZB4axdMHGwEaQjkZzFjQiN9LVYvIFB2nSUHW9tYpxWriPrWDASIxiaXax83REcLxuSdnGPZtw==
-"@budibase/backend-core@1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-1.2.5.tgz#4d8e9e86587f73244c298c4483715b5754e1eebd"
- integrity sha512-8HChhwa6Mo4FouNAWtai5F9FugGCg5EQ8nn47/W5O+WM/VaWdJcjGURHfR9KnWHQUOyWlonvhHEgzyFWp5t+7w==
+"@budibase/backend-core@1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/backend-core/-/backend-core-1.2.14.tgz#1aca6a82b8ac28f5c101721ba4596f674eb57197"
+ integrity sha512-870ZzNX13/f1U7qkUJUoKp/e8UQqDHEc0TZBBaWcpLXbEBs6ikxGp5YdYGEVXkr/ca4rusfHs6jAjUNkIPThow==
dependencies:
- "@budibase/types" "^1.2.5"
+ "@budibase/types" "^1.2.14"
"@techpass/passport-openidconnect" "0.3.2"
aws-sdk "2.1030.0"
bcrypt "5.0.1"
@@ -324,21 +324,21 @@
uuid "8.3.2"
zlib "1.0.5"
-"@budibase/pro@1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-1.2.5.tgz#3a0c2adc994027e114bb133ac26f0562bdb7c945"
- integrity sha512-//RB2p5FVkhH8ylMFyaU0Z1Gh4d6x9p4lm5wabe282O1sIdBAbsq9mtMK6NIftZgakELHS/xwOR6W/knKB4Ybw==
+"@budibase/pro@1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/pro/-/pro-1.2.14.tgz#bce72c8a0d48dbec8dc6aa4af47abb9dbfd3d029"
+ integrity sha512-nYyVHO1IaoDRFSrLp0ueKWG0/Z+PSRs3yF+OZ7vQaECCXXSgWnZyCOy9o6iIovUlbimbPxrQvkwcL8v4deWuvQ==
dependencies:
- "@budibase/backend-core" "1.2.5"
- "@budibase/types" "1.2.5"
+ "@budibase/backend-core" "1.2.14"
+ "@budibase/types" "1.2.14"
"@koa/router" "8.0.8"
joi "17.6.0"
node-fetch "^2.6.1"
-"@budibase/types@1.2.5", "@budibase/types@^1.2.5":
- version "1.2.5"
- resolved "https://registry.yarnpkg.com/@budibase/types/-/types-1.2.5.tgz#0d4dafdee42fb5324dba9393206c6e34d4538afe"
- integrity sha512-ZEDy3TlVSWEhQy7HkAqsuZWJmFxuELo6pu77mpH2fEHY+UpGkSSQPxMg7LYYuWDNsGjJGl0urXh0G5StDM1aTw==
+"@budibase/types@1.2.14", "@budibase/types@^1.2.14":
+ version "1.2.14"
+ resolved "https://registry.yarnpkg.com/@budibase/types/-/types-1.2.14.tgz#8ef2d1b526661629dd9db2f10fea515fadfdcc53"
+ integrity sha512-E3k1kJz24W0l1aw1cL2Edf7WnO6Nt6iew4stDpuQjG2r1EcXPv6/hPsRA7Ms0331ZjGKEswaddSyF5NTn1O8VQ==
"@cspotcode/source-map-consumer@0.8.0":
version "0.8.0"