Merge remote-tracking branch 'origin/develop' into feature/smtp-automation-check

README.md

@@ -8,18 +8,19 @@
 </h1>
 
 <h3 align="center">
-  Build, automate and self-host internal tools in minutes
+  The low code platform you'll enjoy using
 </h3>
 <p align="center">
-  Budibase is an open-source low-code platform, helping developers and IT professionals build, automate, and ship internal tools on their own infrastructure in minutes.
+  Budibase is an open source low-code platform, and the easiest way to build internal tools that improve productivity.
 </p>
 
 <h3 align="center">
  🤖 🎨 🚀
 </h3>
+<br>
 
 <p align="center">
-  <img alt="Budibase design ui" src="https://i.imgur.com/5BnXPsN.png">
+  <img alt="Budibase design ui" src="https://res.cloudinary.com/daog6scxm/image/upload/v1633524049/ui/design-ui-wide-mobile_gdaveq.jpg">
 </p>
 
 <p align="center">
@@ -65,68 +66,25 @@
 
 - **Admin paradise.** Budibase is made to scale. With Budibase, you can self-host on your own infrastructure and globally manage users, onboarding, SMTP, apps, groups, theming and more. You can also provide users/groups with an app portal and disseminate user-management to the group manager.
 
-<br />
-
----
-
-<br />
+<br /><br /><br />
 
 ## 🏁 Get started
-Currently there are two ways to get started with Budibase; Digital Ocean, and Docker.
+
+<img src="https://res.cloudinary.com/daog6scxm/image/upload/v1634808888/logo/deploy_npl9za.png" />
+
+Deploy Budibase self-Hosted in your existing infrastructure, using Docker, Kubernetes, and Digital Ocean.
+Or use Budibase Cloud if you don't need to self-host, and would like to get started quickly.
+
+### [Get started with Budibase](https://budibase.com)
+
+
 <br /><br />
 
-### Get started with Digital Ocean
-The easiest and quickest way to get started, is to use Digital Ocean:
-<a href="https://marketplace.digitalocean.com/apps/budibase">1-click Digital Ocean deploy</a>
-
-<a href="https://marketplace.digitalocean.com/apps/budibase">
-  <img src="https://user-images.githubusercontent.com/552074/87779219-5c3b7600-c824-11ea-9898-981a8ba94f6c.png" alt="digital ocean badge">
-</a>  
-<br /><br />
-
-### Get started with Docker
-To get started, you must have docker and docker compose installed on your machine. 
-Once you have Docker installed, the process takes 5 minutes, with these four steps:
-
-1. Install the Budibase CLI.
-
-```
-$ npm i -g @budibase/cli 
-```
-
-
-2. Setup Budibase (select where to store Budibase, and the port to run it on)
-
-```
-budi hosting --init 
-```
-
-
-3. Run Budibase
-
-```
-budi hosting --start 
-```
-
-
-4. Create your admin user
-
-Enter the email and password for the new admin user.
-
-Done! You are now ready to build powerful internal tools in minutes. For additional information on how to get started and learn Budibase, visit our [docs](https://docs.budibase.com/getting-started).
-
-<br />
-
----
-
-<br />
-
 ## 🎓 Learning Budibase
 
 The Budibase documentation [lives here](https://docs.budibase.com).
 <br />
 
----
 
 <br /><br />
 
@@ -134,22 +92,17 @@ The Budibase documentation [lives here](https://docs.budibase.com).
 
 If you have a question or would like to talk with other Budibase users and join our community, please hop over to [Github discussions](https://github.com/Budibase/budibase/discussions)
 
-<img src="https://d33wubrfki0l68.cloudfront.net/e9241201fd89f9abbbdaac4fe44bb16312752abe/84013/img/hero-images/community.webp" />
+<br /><br /><br />
 
-<br /><br />
-
----
-
-<br />
 
 ## ❗ Code of conduct
 
 Budibase is dedicated to providing a welcoming, diverse, and harrassment-free experience for everyone. We expect everyone in the Budibase community to abide by our [**Code of Conduct**](https://github.com/Budibase/budibase/blob/HEAD/.github/CODE_OF_CONDUCT.md). Please read it.
 <br />
 
----
 
-<br />
+<br /><br />
+
 
 ## 🙌 Contributing to Budibase
 
@@ -168,32 +121,22 @@ Budibase is a monorepo managed by lerna. Lerna manages the building and publishi
 - [packages/server](https://github.com/Budibase/budibase/tree/HEAD/packages/server) - The budibase server. This Koa app is responsible for serving the JS for the builder and budibase apps, as well as providing the API for interaction with the database and file system.
 
 For more information, see [CONTRIBUTING.md](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md)
-<br /><br />
-
----
 
 <br /><br />
 
+
 ## 📝 License
 
 Budibase is open-source, licensed as [GPL v3](https://www.gnu.org/licenses/gpl-3.0.en.html). The client and component libraries are licensed as [MPL](https://directory.fsf.org/wiki/License:MPL-2.0) - so the apps that you build can be licensed however you like.
+
 <br /><br />
 
----
-
-
-<br />
-
 ## ⭐ Stargazers over time
 
 [![Stargazers over time](https://starchart.cc/Budibase/budibase.svg)](https://starchart.cc/Budibase/budibase)
 
 If you are having issues between updates of the builder, please use the guide [here](https://github.com/Budibase/budibase/blob/HEAD/.github/CONTRIBUTING.md#troubleshooting) to clear down your environment.
 
-<br />
-
----
-
 <br /><br />
 
 ## Contributors ✨

hosting/envoy.dev.yaml.hbs

@@ -41,6 +41,7 @@ static_resources:
               - match: { prefix: "/api/" }
                 route:
                   cluster: server-dev
+                  timeout: 120s
 
               - match: { prefix: "/app_" }
                 route:

hosting/envoy.yaml

@@ -58,6 +58,7 @@ static_resources:
               - match: { prefix: "/api/" }
                 route:
                   cluster: app-service
+                  timeout: 120s
 
               - match: { prefix: "/worker/" }
                 route: 

lerna.json

@@ -1,5 +1,5 @@
 {
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "npmClient": "yarn",
   "packages": [
     "packages/*"

packages/auth/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@budibase/auth",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "description": "Authentication middlewares for budibase builder and apps",
   "main": "src/index.js",
   "author": "Budibase",

packages/bbui/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@budibase/bbui",
   "description": "A UI solution used in the different Budibase projects.",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "license": "AGPL-3.0",
   "svelte": "src/index.js",
   "module": "dist/bbui.es.js",

packages/bbui/src/Form/Core/DatePicker.svelte

@@ -31,7 +31,11 @@
 
   const handleChange = event => {
     const [dates] = event.detail
-    dispatch("change", dates[0])
+    let newValue = dates[0]
+    if (newValue) {
+      newValue = newValue.toISOString()
+    }
+    dispatch("change", newValue)
   }
 
   const clearDateOnBackspace = event => {
@@ -57,11 +61,38 @@
     const els = document.querySelectorAll(`#${flatpickrId} input`)
     els.forEach(el => el.blur())
   }
+
+  const parseDate = val => {
+    if (!val) {
+      return null
+    }
+    let date
+    if (val instanceof Date) {
+      // Use real date obj if already parsed
+      date = val
+    } else if (isNaN(val)) {
+      // Treat as date string of some sort
+      date = new Date(val)
+    } else {
+      // Treat as numerical timestamp
+      date = new Date(parseInt(val))
+    }
+    const time = date.getTime()
+    if (isNaN(time)) {
+      return null
+    }
+    // By rounding to the nearest second we avoid locking up in an endless
+    // loop in the builder, caused by potentially enriching {{ now }} to every
+    // millisecond.
+    return new Date(Math.floor(time / 1000) * 1000)
+  }
+
+  $: console.log(value)
 </script>
 
 <Flatpickr
   bind:flatpickr
-  {value}
+  value={parseDate(value)}
   on:open={onOpen}
   on:close={onClose}
   options={flatpickrOptions}

packages/bbui/src/Form/DatePicker.svelte

@@ -13,10 +13,10 @@
   export let appendTo = undefined
 
   const dispatch = createEventDispatcher()
+
   const onChange = e => {
-    const isoString = e.detail.toISOString()
-    value = isoString
-    dispatch("change", isoString)
+    value = e.detail
+    dispatch("change", e.detail)
   }
 </script>
 

packages/builder/cypress/integration/customThemingProperties.spec.js

@@ -5,12 +5,12 @@ context("Custom Theming Properties", () => {
       cy.navigateToFrontend()
     })
 
-    // Default Values
-    // Button roundness = Large
-    // Accent colour = Blue 600
-    // Accent colour (hover) = Blue 500
-    // Navigation bar background colour = Gray 100
-    // Navigation bar text colour = Gray 800
+    /* Default Values:
+    Button roundness = Large
+    Accent colour = Blue 600
+    Accent colour (hover) = Blue 500
+    Navigation bar background colour = Gray 100
+    Navigation bar text colour = Gray 800 */
     it("should reset the color property values", () => {
       // Open Theme modal and change colours  
       cy.get(".spectrum-ActionButton-label").contains("Theme").click()
@@ -24,6 +24,29 @@ context("Custom Theming Properties", () => {
         checkThemeColorDefaults()
     })
     
+    /* Button Roundness Values:
+    None = 0
+    Small = 4px
+    Medium = 8px
+    Large = 16px */
+    it("should test button roundness", () => {
+      const buttonRoundnessValues = ["0", "4px", "8px", "16px"]
+      cy.wait(1000)
+      // Add button, change roundness and confirm value
+      cy.addComponent("Button", null).then((componentId) => {
+        buttonRoundnessValues.forEach(function (item, index){
+          cy.get(".spectrum-ActionButton-label").contains("Theme").click()
+          cy.get(".setting").contains("Button roundness").parent()
+          .get(".select-wrapper").click()
+          cy.get(".spectrum-Popover").find('li').eq(index).click()
+          cy.get(".spectrum-Button").contains("View changes").click({force: true})
+          cy.reload()
+          cy.getComponent(componentId)
+        .parents(".svelte-xiqd1c").eq(0).should('have.attr', 'style').and('contains', `--buttonBorderRadius:${item}`)
+      })
+    })
+  })
+    
     const changeThemeColors = () => {
       // Changes the theme colours
       cy.get(".spectrum-FieldLabel").contains("Accent color")

packages/builder/cypress/integration/renameAnApplication.spec.js

@@ -0,0 +1,102 @@
+context("Rename an App", () => {
+    beforeEach(() => {
+      cy.login()
+      cy.createTestApp()
+    })
+
+it("should rename an unpublished application", () => {
+    const appRename = "Cypress Renamed"
+    // Rename app, Search for app, Confirm name was changed
+    cy.get(".home-logo").click()
+    renameApp(appRename)
+    cy.searchForApplication(appRename)
+    cy.get(".appGrid").find(".wrapper").should("have.length", 1)
+    })
+    
+it("Should rename a published application", () => {
+    // It is not possible to rename a published application
+    const appRename = "Cypress Renamed"
+    // Publish the app
+    cy.get(".toprightnav")
+    cy.get(".spectrum-Button").contains("Publish").click({force: true})
+    cy.get(".spectrum-Dialog-grid")
+        .within(() => {
+            // Click publish again within the modal
+            cy.get(".spectrum-Button").contains("Publish").click({force: true})
+        })
+    // Rename app, Search for app, Confirm name was changed
+    cy.get(".home-logo").click()
+    renameApp(appRename, true)
+    cy.searchForApplication(appRename)
+    cy.get(".appGrid").find(".wrapper").should("have.length", 1)
+})
+
+it("Should try to rename an application to have no name", () => {
+    cy.get(".home-logo").click()
+    renameApp(" ", false, true)
+    // Close modal and confirm name has not been changed
+    cy.get(".spectrum-Dialog-grid").contains("Cancel").click()
+    cy.searchForApplication("Cypress Tests")
+    cy.get(".appGrid").find(".wrapper").should("have.length", 1)
+})
+
+it("Should create two applications with the same name", () => {
+    // It is not possible to have applications with the same name
+    const appName = "Cypress Tests"
+    cy.visit(`localhost:${Cypress.env("PORT")}/builder`)
+    cy.wait(500)
+    cy.get(".spectrum-Button").contains("Create app").click({force: true})
+    cy.contains(/Start from scratch/).click()
+    cy.get(".spectrum-Modal")
+    .within(() => {
+      cy.get("input").eq(0).type(appName)
+      cy.get(".spectrum-ButtonGroup").contains("Create app").click({force: true})
+      cy.get(".error").should("have.text", "Another app with the same name already exists")      
+    })
+})
+
+it("should validate application names", () => {
+    // App name must be letters, numbers and spaces only
+    // This test checks numbers and special characters specifically
+    const numberName = 12345
+    const specialCharName = "£$%^"
+    cy.get(".home-logo").click()
+    renameApp(numberName)
+    cy.searchForApplication(numberName)
+    cy.get(".appGrid").find(".wrapper").should("have.length", 1)
+    renameApp(specialCharName)
+    cy.get(".error").should("have.text", "App name must be letters, numbers and spaces only")
+})
+    
+    const renameApp = (appName, published, noName) => {
+    cy.request(`localhost:${Cypress.env("PORT")}/api/applications?status=all`)
+    .its("body")
+    .then(val => {
+        if (val.length > 0) {
+            cy.get(".title > :nth-child(3) > .spectrum-Icon").click()
+            // Check for when an app is published
+            if (published == true){
+                // Should not have Edit as option, will unpublish app
+                cy.should("not.have.value", "Edit")
+                cy.get(".spectrum-Menu").contains("Unpublish").click()
+                cy.get(".spectrum-Dialog-grid").contains("Unpublish app").click()
+                cy.get(".title > :nth-child(3) > .spectrum-Icon").click()
+            }
+            cy.contains("Edit").click()
+            cy.get(".spectrum-Modal")
+            .within(() => {
+                if (noName == true){
+                    cy.get("input").clear()
+                    cy.get(".spectrum-Dialog-grid").click()
+                    .contains("App name must be letters, numbers and spaces only")
+                    return cy
+                }
+                cy.get("input").clear()
+                cy.get("input").eq(0).type(appName).should("have.value", appName).blur()
+                cy.get(".spectrum-ButtonGroup").contains("Save").click({force: true})
+                cy.wait(500)
+      })
+    }
+    })
+}
+})

packages/builder/cypress/support/commands.js

@@ -35,7 +35,7 @@ Cypress.Commands.add("login", () => {
 Cypress.Commands.add("createApp", name => {
   cy.visit(`localhost:${Cypress.env("PORT")}/builder`)
   cy.wait(500)
-  cy.contains(/Start from scratch/).click()
+  cy.contains(/Start from scratch/).dblclick()
   cy.get(".spectrum-Modal").within(() => {
     cy.get("input").eq(0).type(name).should("have.value", name).blur()
     cy.get(".spectrum-ButtonGroup").contains("Create app").click()
@@ -224,3 +224,9 @@ Cypress.Commands.add("addCustomSourceOptions", totalOptions => {
       cy.get(".spectrum-Button").contains("Save").click({ force: true })
     })
 })
+
+Cypress.Commands.add("searchForApplication", appName => {
+  cy.get(".spectrum-Textfield").within(() => {
+    cy.get("input").eq(0).type(appName)
+  })
+})

packages/builder/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@budibase/builder",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "license": "AGPL-3.0",
   "private": true,
   "scripts": {
@@ -65,10 +65,10 @@
     }
   },
   "dependencies": {
-    "@budibase/bbui": "^0.9.169-alpha.1",
-    "@budibase/client": "^0.9.169-alpha.1",
+    "@budibase/bbui": "^0.9.169-alpha.11",
+    "@budibase/client": "^0.9.169-alpha.11",
     "@budibase/colorpicker": "1.1.2",
-    "@budibase/string-templates": "^0.9.169-alpha.1",
+    "@budibase/string-templates": "^0.9.169-alpha.11",
     "@sentry/browser": "6.0.0",
     "@spectrum-css/page": "^3.0.1",
     "@spectrum-css/vars": "^3.0.1",

packages/builder/src/builderStore/dataBinding.js

@@ -472,7 +472,7 @@ function bindingReplacement(bindableProperties, textWithBindings, convertTo) {
           idx = searchString.indexOf(from)
           if (idx !== -1) {
             let end = idx + from.length,
-              searchReplace = Array(binding[convertTo].length).join("*")
+              searchReplace = Array(binding[convertTo].length + 1).join("*")
             // blank out parts of the search string
             searchString = replaceBetween(searchString, idx, end, searchReplace)
             newBoundValue = replaceBetween(

packages/builder/src/components/backend/DataTable/modals/FilterModal.svelte

@@ -82,34 +82,30 @@
 
   function isMultipleChoice(field) {
     return (
-      (viewTable.schema[field].constraints &&
-        viewTable.schema[field].constraints.inclusion &&
-        viewTable.schema[field].constraints.inclusion.length) ||
-      viewTable.schema[field].type === "boolean"
+      viewTable.schema[field]?.constraints?.inclusion?.length ||
+      viewTable.schema[field]?.type === "boolean"
     )
   }
 
   function fieldOptions(field) {
-    return viewTable.schema[field].type === "options"
-      ? viewTable.schema[field].constraints.inclusion
+    return viewTable.schema[field]?.type === "options"
+      ? viewTable.schema[field]?.constraints.inclusion
       : [true, false]
   }
 
   function isDate(field) {
-    return viewTable.schema[field].type === "datetime"
+    return viewTable.schema[field]?.type === "datetime"
   }
 
   function isNumber(field) {
-    return viewTable.schema[field].type === "number"
+    return viewTable.schema[field]?.type === "number"
   }
 
   const fieldChanged = filter => ev => {
-    // reset if type changed
-    if (
-      filter.key &&
-      ev.detail &&
-      viewTable.schema[filter.key].type !== viewTable.schema[ev.detail].type
-    ) {
+    // Reset if type changed
+    const oldType = viewTable.schema[filter.key]?.type
+    const newType = viewTable.schema[ev.detail]?.type
+    if (filter.key && ev.detail && oldType !== newType) {
       filter.value = ""
     }
   }

packages/builder/src/components/start/CreateAppModal.svelte

@@ -20,6 +20,7 @@
   import TemplateList from "./TemplateList.svelte"
 
   export let template
+  export let inline
 
   const values = writable({ name: null })
   const errors = writable({})
@@ -39,9 +40,10 @@
 
   let submitting = false
   let valid = false
+  let initialTemplateInfo = template?.fromFile || template?.key
 
   $: checkValidity($values, validator)
-  $: showTemplateSelection = !template?.fromFile && !template?.key
+  $: showTemplateSelection = !template && !initialTemplateInfo
 
   onMount(async () => {
     await hostingStore.actions.fetchDeployedApps()
@@ -64,6 +66,11 @@
   const checkValidity = async (values, validator) => {
     const obj = object().shape(validator)
     Object.keys(validator).forEach(key => ($errors[key] = null))
+    if (template?.fromFile && values.file == null) {
+      valid = false
+      return
+    }
+
     try {
       await obj.validate(values, { abortEarly: false })
     } catch (validationErrors) {
@@ -71,14 +78,17 @@
         $errors[error.path] = capitalise(error.message)
       })
     }
+
     valid = await obj.isValid(values)
   }
 
   async function createNewApp() {
+    const letTemplateToUse =
+      Object.keys(template).length === 0 ? null : template
     submitting = true
 
     // Check a template exists if we are important
-    if (template?.fromFile && !$values.file) {
+    if (letTemplateToUse?.fromFile && !$values.file) {
       $errors.file = "Please choose a file to import"
       valid = false
       submitting = false
@@ -89,10 +99,10 @@
       // Create form data to create app
       let data = new FormData()
       data.append("name", $values.name.trim())
-      data.append("useTemplate", template != null)
-      if (template) {
-        data.append("templateName", template.name)
-        data.append("templateKey", template.key)
+      data.append("useTemplate", letTemplateToUse != null)
+      if (letTemplateToUse) {
+        data.append("templateName", letTemplateToUse.name)
+        data.append("templateKey", letTemplateToUse.key)
         data.append("templateFile", $values.file)
       }
 
@@ -106,7 +116,7 @@
       analytics.captureEvent(Events.APP.CREATED, {
         name: $values.name,
         appId: appJson.instance._id,
-        template,
+        letTemplateToUse,
       })
 
       // Select Correct Application/DB in prep for creating user
@@ -144,19 +154,18 @@
     showConfirmButton={false}
     size="L"
     onConfirm={() => {
-      showTemplateSelection = false
+      template = {}
       return false
     }}
-    showCancelButton={false}
-    showCloseIcon={false}
+    showCancelButton={!inline}
+    showCloseIcon={!inline}
   >
     <TemplateList
-      onSelect={selected => {
+      onSelect={(selected, { useImport } = {}) => {
         if (!selected) {
-          showTemplateSelection = false
+          template = useImport ? { fromFile: true } : {}
           return
         }
-
         template = selected
       }}
     />
@@ -166,6 +175,9 @@
     title={template?.fromFile ? "Import app" : "Create app"}
     confirmText={template?.fromFile ? "Import app" : "Create app"}
     onConfirm={createNewApp}
+    onCancel={inline ? () => (template = null) : null}
+    cancelText={inline ? "Back" : undefined}
+    showCloseIcon={!inline}
     disabled={!valid}
   >
     {#if template?.fromFile}

packages/builder/src/components/start/TemplateList.svelte

@@ -7,7 +7,6 @@
 
   async function fetchTemplates() {
     const response = await api.get("/api/templates?type=app")
-    console.log("Responded")
     return await response.json()
   }
 
@@ -48,6 +47,19 @@
         <Heading size="XS">Start from scratch</Heading>
         <p class="detail">BLANK</p>
       </div>
+      <div
+        class="template import"
+        on:click={() => onSelect(null, { useImport: true })}
+      >
+        <div
+          class="background-icon"
+          style={`background: rgb(50, 50, 50); color: white;`}
+        >
+          <Icon name="Add" />
+        </div>
+        <Heading size="XS">Import an app</Heading>
+        <p class="detail">BLANK</p>
+      </div>
     </div>
   {:catch err}
     <h1 style="color:red">{err}</h1>
@@ -95,4 +107,8 @@
     background: var(--spectrum-global-color-gray-50);
     margin-top: 20px;
   }
+
+  .import {
+    background: var(--spectrum-global-color-gray-50);
+  }
 </style>

packages/builder/src/pages/builder/portal/apps/index.svelte

@@ -112,16 +112,8 @@
 
   const exportApp = app => {
     const id = app.deployed ? app.prodId : app.devId
-    try {
-      download(
-        `/api/backups/export?appId=${id}&appname=${encodeURIComponent(
-          app.name
-        )}`
-      )
-      notifications.success("App exported successfully")
-    } catch (err) {
-      notifications.error(`Error exporting app: ${err}`)
-    }
+    const appName = encodeURIComponent(app.name)
+    window.location = `/api/backups/export?appId=${id}&appname=${appName}`
   }
 
   const unpublishApp = app => {
@@ -268,7 +260,7 @@
   {#if !enrichedApps.length && !creatingApp && loaded}
     <div class="empty-wrapper">
       <Modal inline>
-        <CreateAppModal {template} />
+        <CreateAppModal {template} inline={true} />
       </Modal>
     </div>
   {/if}

packages/cli/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@budibase/cli",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "description": "Budibase CLI, for developers, self hosting and migrations.",
   "main": "src/index.js",
   "bin": {

packages/client/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@budibase/client",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "license": "MPL-2.0",
   "module": "dist/budibase-client.js",
   "main": "dist/budibase-client.js",
@@ -19,9 +19,9 @@
     "dev:builder": "rollup -cw"
   },
   "dependencies": {
-    "@budibase/bbui": "^0.9.169-alpha.1",
+    "@budibase/bbui": "^0.9.169-alpha.11",
     "@budibase/standard-components": "^0.9.139",
-    "@budibase/string-templates": "^0.9.169-alpha.1",
+    "@budibase/string-templates": "^0.9.169-alpha.11",
     "regexparam": "^1.3.0",
     "shortid": "^2.2.15",
     "svelte-spa-router": "^3.0.5"

packages/client/src/components/ClientApp.svelte

@@ -113,6 +113,13 @@
                     />
                   {/key}
 
+                  <!--
+                    Flatpickr needs to be inside the theme wrapper.
+                    It also needs its own container because otherwise it hijacks
+                    key events on the whole page. It is painful to work with.
+                  -->
+                  <div id="flatpickr-root" />
+
                   <!-- Layers on top of app -->
                   <NotificationDisplay />
                   <ConfirmationDisplay />

packages/client/src/components/app/forms/DateTimeField.svelte

@@ -12,31 +12,6 @@
 
   let fieldState
   let fieldApi
-
-  const parseDate = val => {
-    if (!val) {
-      return null
-    }
-    let date
-    if (val instanceof Date) {
-      // Use real date obj if already parsed
-      date = val
-    } else if (isNaN(val)) {
-      // Treat as date string of some sort
-      date = new Date(val)
-    } else {
-      // Treat as numerical timestamp
-      date = new Date(parseInt(val))
-    }
-    const time = date.getTime()
-    if (isNaN(time)) {
-      return null
-    }
-    // By rounding to the nearest second we avoid locking up in an endless
-    // loop in the builder, caused by potentially enriching {{ now }} to every
-    // millisecond.
-    return new Date(Math.floor(time / 1000) * 1000)
-  }
 </script>
 
 <Field
@@ -44,7 +19,7 @@
   {field}
   {disabled}
   {validation}
-  defaultValue={parseDate(defaultValue)}
+  {defaultValue}
   type="datetime"
   bind:fieldState
   bind:fieldApi
@@ -56,7 +31,7 @@
       disabled={fieldState.disabled}
       error={fieldState.error}
       id={fieldState.fieldId}
-      appendTo={document.getElementById("theme-root")}
+      appendTo={document.getElementById("flatpickr-root")}
       {enableTime}
       {placeholder}
     />

packages/server/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@budibase/server",
   "email": "hi@budibase.com",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "description": "Budibase Web Server",
   "main": "src/index.js",
   "repository": {
@@ -68,9 +68,9 @@
   "author": "Budibase",
   "license": "AGPL-3.0-or-later",
   "dependencies": {
-    "@budibase/auth": "^0.9.169-alpha.1",
-    "@budibase/client": "^0.9.169-alpha.1",
-    "@budibase/string-templates": "^0.9.169-alpha.1",
+    "@budibase/auth": "^0.9.169-alpha.11",
+    "@budibase/client": "^0.9.169-alpha.11",
+    "@budibase/string-templates": "^0.9.169-alpha.11",
     "@elastic/elasticsearch": "7.10.0",
     "@koa/router": "8.0.0",
     "@sendgrid/mail": "7.1.1",

packages/server/src/api/controllers/backup.js

@@ -1,10 +1,9 @@
-const { performBackup } = require("../../utilities/fileSystem")
+const { streamBackup } = require("../../utilities/fileSystem")
 
 exports.exportAppDump = async function (ctx) {
   const { appId } = ctx.query
-  const appname = decodeURI(ctx.query.appname)
-  const backupIdentifier = `${appname}Backup${new Date().getTime()}.txt`
-
+  const appName = decodeURI(ctx.query.appname)
+  const backupIdentifier = `${appName}-export-${new Date().getTime()}.txt`
   ctx.attachment(backupIdentifier)
-  ctx.body = await performBackup(appId, backupIdentifier)
+  ctx.body = await streamBackup(appId)
 }

packages/server/src/api/controllers/permission.js

@@ -147,6 +147,7 @@ exports.getResourcePerms = async function (ctx) {
       const rolePerms = role.permissions
       if (
         rolePerms &&
+        rolePerms[resourceId] &&
         (rolePerms[resourceId] === level ||
           rolePerms[resourceId].indexOf(level) !== -1)
       ) {

packages/server/src/api/controllers/row/ExternalRequest.ts

@@ -15,8 +15,9 @@ import {
 import {
   breakRowIdField,
   generateRowIdField,
+  isRowId,
+  convertRowId,
 } from "../../../integrations/utils"
-import { RelationshipTypes } from "../../../constants"
 
 interface ManyRelationship {
   tableId?: string
@@ -36,7 +37,7 @@ interface RunConfig {
 
 module External {
   const { makeExternalQuery } = require("./utils")
-  const { DataSourceOperation, FieldTypes } = require("../../../constants")
+  const { DataSourceOperation, FieldTypes, RelationshipTypes } = require("../../../constants")
   const { breakExternalTableId, isSQL } = require("../../../integrations/utils")
   const { processObjectSync } = require("@budibase/string-templates")
   const { cloneDeep } = require("lodash/fp")
@@ -83,6 +84,48 @@ module External {
     }
   }
 
+  /**
+   * This function checks the incoming parameters to make sure all the inputs are
+   * valid based on on the table schema. The main thing this is looking for is when a
+   * user has made use of the _id field of a row for a foreign key or a search parameter.
+   * In these cases the key will be sent up as [1], rather than 1. In these cases we will
+   * simplify it down to the requirements. This function is quite complex as we try to be
+   * relatively restrictive over what types of columns we will perform this action for.
+   */
+  function cleanupConfig(config: RunConfig, table: Table): RunConfig {
+    const primaryOptions = [
+      FieldTypes.STRING,
+      FieldTypes.LONGFORM,
+      FieldTypes.OPTIONS,
+      FieldTypes.NUMBER,
+    ]
+    // filter out fields which cannot be keys
+    const fieldNames = Object.entries(table.schema)
+      .filter(schema => primaryOptions.find(val => val === schema[1].type))
+      .map(([fieldName]) => fieldName)
+    const iterateObject = (obj: { [key: string]: any }) => {
+      for (let [field, value] of Object.entries(obj)) {
+        if (fieldNames.find(name => name === field) && isRowId(value)) {
+          obj[field] = convertRowId(value)
+        }
+      }
+    }
+    // check the row and filters to make sure they aren't a key of some sort
+    if (config.filters) {
+      for (let filter of Object.values(config.filters)) {
+        if (typeof filter !== "object" || Object.keys(filter).length === 0) {
+          continue
+        }
+        iterateObject(filter)
+      }
+    }
+    if (config.row) {
+      iterateObject(config.row)
+    }
+
+    return config
+  }
+
   function generateIdForRow(row: Row | undefined, table: Table): string {
     const primary = table.primary
     if (!row || !primary) {
@@ -509,7 +552,7 @@ module External {
       return fields
     }
 
-    async run({ id, row, filters, sort, paginate }: RunConfig) {
+    async run(config: RunConfig) {
       const { appId, operation, tableId } = this
       let { datasourceId, tableName } = breakExternalTableId(tableId)
       if (!this.datasource) {
@@ -525,9 +568,11 @@ module External {
       if (!table) {
         throw `Unable to process query, table "${tableName}" not defined.`
       }
-      // clean up row on ingress using schema
+      // look for specific components of config which may not be considered acceptable
+      let { id, row, filters, sort, paginate } = cleanupConfig(config, table)
       filters = buildFilters(id, filters || {}, table)
       const relationships = this.buildRelationships(table)
+      // clean up row on ingress using schema
       const processed = this.inputProcessing(row, table)
       row = processed.row
       if (

packages/server/src/api/controllers/table/index.js

@@ -9,11 +9,7 @@ const {
   BudibaseInternalDB,
 } = require("../../../db/utils")
 const { FieldTypes } = require("../../../constants")
-const { TableSaveFunctions, getExternalTable } = require("./utils")
-const {
-  isExternalTable,
-  breakExternalTableId,
-} = require("../../../integrations/utils")
+const { TableSaveFunctions, getTable } = require("./utils")
 
 exports.fetch = async function (ctx) {
   const db = new CouchDB(ctx.appId)
@@ -48,14 +44,8 @@ exports.fetch = async function (ctx) {
 }
 
 exports.find = async function (ctx) {
-  const db = new CouchDB(ctx.appId)
   const tableId = ctx.params.id
-  if (isExternalTable(tableId)) {
-    let { datasourceId, tableName } = breakExternalTableId(tableId)
-    ctx.body = await getExternalTable(ctx.appId, datasourceId, tableName)
-  } else {
-    ctx.body = await db.get(ctx.params.id)
-  }
+  ctx.body = await getTable(ctx.appId, tableId)
 }
 
 exports.save = async function (ctx) {

packages/server/src/api/controllers/table/utils.js

@@ -9,6 +9,12 @@ const { isEqual } = require("lodash/fp")
 const { AutoFieldSubTypes, FieldTypes } = require("../../../constants")
 const { inputProcessing } = require("../../../utilities/rowProcessor")
 const { USERS_TABLE_SCHEMA } = require("../../../constants")
+const {
+  isExternalTable,
+  breakExternalTableId,
+} = require("../../../integrations/utils")
+const { getViews, saveView } = require("../view/utils")
+const viewTemplate = require("../view/viewBuilder")
 
 exports.checkForColumnUpdates = async (db, oldTable, updatedTable) => {
   let updatedRows = []
@@ -21,6 +27,7 @@ exports.checkForColumnUpdates = async (db, oldTable, updatedTable) => {
   }
   // check for renaming of columns or deleted columns
   if (rename || deletedColumns.length !== 0) {
+    // Update all rows
     const rows = await db.allDocs(
       getRowParams(updatedTable._id, null, {
         include_docs: true,
@@ -35,6 +42,9 @@ exports.checkForColumnUpdates = async (db, oldTable, updatedTable) => {
       }
       return doc
     })
+
+    // Update views
+    await exports.checkForViewUpdates(db, updatedTable, rename, deletedColumns)
     delete updatedTable._rename
   }
   return { rows: updatedRows, table: updatedTable }
@@ -223,4 +233,86 @@ exports.getExternalTable = async (appId, datasourceId, tableName) => {
   return entities[tableName]
 }
 
+exports.getTable = async (appId, tableId) => {
+  const db = new CouchDB(appId)
+  if (isExternalTable(tableId)) {
+    let { datasourceId, tableName } = breakExternalTableId(tableId)
+    return exports.getExternalTable(appId, datasourceId, tableName)
+  } else {
+    return db.get(tableId)
+  }
+}
+
+exports.checkForViewUpdates = async (db, table, rename, deletedColumns) => {
+  const views = await getViews(db)
+  const tableViews = views.filter(view => view.meta.tableId === table._id)
+
+  // Check each table view to see if impacted by this table action
+  for (let view of tableViews) {
+    let needsUpdated = false
+
+    // First check for renames, otherwise check for deletions
+    if (rename) {
+      // Update calculation field if required
+      if (view.meta.field === rename.old) {
+        view.meta.field = rename.updated
+        needsUpdated = true
+      }
+
+      // Update group by field if required
+      if (view.meta.groupBy === rename.old) {
+        view.meta.groupBy = rename.updated
+        needsUpdated = true
+      }
+
+      // Update filters if required
+      if (view.meta.filters) {
+        view.meta.filters.forEach(filter => {
+          if (filter.key === rename.old) {
+            filter.key = rename.updated
+            needsUpdated = true
+          }
+        })
+      }
+    } else if (deletedColumns) {
+      deletedColumns.forEach(column => {
+        // Remove calculation statement if required
+        if (view.meta.field === column) {
+          delete view.meta.field
+          delete view.meta.calculation
+          delete view.meta.groupBy
+          needsUpdated = true
+        }
+
+        // Remove group by field if required
+        if (view.meta.groupBy === column) {
+          delete view.meta.groupBy
+          needsUpdated = true
+        }
+
+        // Remove filters referencing deleted field if required
+        if (view.meta.filters && view.meta.filters.length) {
+          const initialLength = view.meta.filters.length
+          view.meta.filters = view.meta.filters.filter(filter => {
+            return filter.key !== column
+          })
+          if (initialLength !== view.meta.filters.length) {
+            needsUpdated = true
+          }
+        }
+      })
+    }
+
+    // Update view if required
+    if (needsUpdated) {
+      const newViewTemplate = viewTemplate(view.meta)
+      await saveView(db, null, view.name, newViewTemplate)
+      if (!newViewTemplate.meta.schema) {
+        newViewTemplate.meta.schema = table.schema
+      }
+      table.views[view.name] = newViewTemplate.meta
+    }
+  }
+}
+
 exports.TableSaveFunctions = TableSaveFunctions

packages/server/src/automations/automationUtils.js

@@ -1,9 +1,4 @@
-const CouchDB = require("../db")
-const {
-  isExternalTable,
-  breakExternalTableId,
-} = require("../integrations/utils")
-const { getExternalTable } = require("../api/controllers/table/utils")
+const { getTable } = require("../api/controllers/table/utils")
 
 /**
  * When values are input to the system generally they will be of type string as this is required for template strings.
@@ -21,7 +16,7 @@ const { getExternalTable } = require("../api/controllers/table/utils")
  * @returns {object} The inputs object which has had all the various types supported by this function converted to their
  * primitive types.
  */
-module.exports.cleanInputValues = (inputs, schema) => {
+exports.cleanInputValues = (inputs, schema) => {
   if (schema == null) {
     return inputs
   }
@@ -63,30 +58,11 @@ module.exports.cleanInputValues = (inputs, schema) => {
  * @param {object} row The input row structure which requires clean-up after having been through template statements.
  * @returns {Promise<Object>} The cleaned up rows object, will should now have all the required primitive types.
  */
-module.exports.cleanUpRow = async (appId, tableId, row) => {
-  const db = new CouchDB(appId)
-  let table
-  if (isExternalTable(tableId)) {
-    const { datasourceId, tableName } = breakExternalTableId(tableId)
-    table = await getExternalTable(appId, datasourceId, tableName)
-  } else {
-    table = await db.get(tableId)
-  }
-
-  return module.exports.cleanInputValues(row, { properties: table.schema })
+exports.cleanUpRow = async (appId, tableId, row) => {
+  let table = await getTable(appId, tableId)
+  return exports.cleanInputValues(row, { properties: table.schema })
 }
 
-/**
- * A utility function for the cleanUpRow, which can be used if only the row ID is known (not the table ID) to clean
- * up a row after template statements have been replaced. This is specifically useful for the update row action.
- *
- * @param {string} appId The instance which the Table/Table is contained under.
- * @param {string} rowId The ID of the row from which the tableId will be extracted, to get the Table/Table schema.
- * @param {object} row The input row structure which requires clean-up after having been through template statements.
- * @returns {Promise<Object>} The cleaned up rows object, which will now have all the required primitive types.
- */
-module.exports.cleanUpRowById = async (appId, rowId, row) => {
-  const db = new CouchDB(appId)
-  const foundRow = await db.get(rowId)
-  return module.exports.cleanUpRow(appId, foundRow.tableId, row)
+exports.getError = err => {
+  return typeof err !== "string" ? err.toString() : err
 }

packages/server/src/automations/steps/bash.js

@@ -1,5 +1,6 @@
 const { execSync } = require("child_process")
 const { processStringSync } = require("@budibase/string-templates")
+const automationUtils = require("../automationUtils")
 
 exports.definition = {
   name: "Bash Scripting",
@@ -63,7 +64,7 @@ exports.run = async function ({ inputs, context }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/createRow.js

@@ -97,7 +97,7 @@ exports.run = async function ({ inputs, appId, emitter }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/deleteRow.js

@@ -2,6 +2,7 @@ const rowController = require("../../api/controllers/row")
 const env = require("../../environment")
 const usage = require("../../utilities/usageQuota")
 const { buildCtx } = require("./utils")
+const automationUtils = require("../automationUtils")
 
 exports.definition = {
   description: "Delete a row from your database",
@@ -85,7 +86,7 @@ exports.run = async function ({ inputs, appId, emitter }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/executeQuery.js

@@ -1,5 +1,6 @@
 const queryController = require("../../api/controllers/query")
 const { buildCtx } = require("./utils")
+const automationUtils = require("../automationUtils")
 
 exports.definition = {
   name: "External Data Connector",
@@ -74,7 +75,7 @@ exports.run = async function ({ inputs, appId, emitter }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/executeScript.js

@@ -1,5 +1,6 @@
 const scriptController = require("../../api/controllers/script")
 const { buildCtx } = require("./utils")
+const automationUtils = require("../automationUtils")
 
 exports.definition = {
   name: "JS Scripting",
@@ -63,7 +64,7 @@ exports.run = async function ({ inputs, appId, context, emitter }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/outgoingWebhook.js

@@ -1,5 +1,6 @@
 const fetch = require("node-fetch")
 const { getFetchResponse } = require("./utils")
+const automationUtils = require("../automationUtils")
 
 const RequestType = {
   POST: "POST",
@@ -127,7 +128,7 @@ exports.run = async function ({ inputs }) {
     /* istanbul ignore next */
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/queryRows.js

@@ -2,6 +2,7 @@ const rowController = require("../../api/controllers/row")
 const tableController = require("../../api/controllers/table")
 const { FieldTypes } = require("../../constants")
 const { buildCtx } = require("./utils")
+const automationUtils = require("../automationUtils")
 
 const SortOrders = {
   ASCENDING: "ascending",
@@ -110,7 +111,7 @@ exports.run = async function ({ inputs, appId }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/sendSmtpEmail.js

@@ -1,4 +1,5 @@
 const { sendSmtpEmail } = require("../../utilities/workerRequests")
+const automationUtils = require("../automationUtils")
 
 exports.definition = {
   description: "Send an email using SMTP",
@@ -61,7 +62,7 @@ exports.run = async function ({ inputs }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/automations/steps/updateRow.js

@@ -64,6 +64,7 @@ exports.run = async function ({ inputs, appId, emitter }) {
       },
     }
   }
+  const tableId = inputs.row.tableId
 
   // clear any falsy properties so that they aren't updated
   for (let propKey of Object.keys(inputs.row)) {
@@ -80,15 +81,14 @@ exports.run = async function ({ inputs, appId, emitter }) {
     },
     params: {
       rowId: inputs.rowId,
+      tableId: tableId,
     },
   })
 
   try {
-    inputs.row = await automationUtils.cleanUpRowById(
-      appId,
-      inputs.rowId,
-      inputs.row
-    )
+    if (tableId) {
+      inputs.row = await automationUtils.cleanUpRow(appId, tableId, inputs.row)
+    }
     await rowController.patch(ctx)
     return {
       row: ctx.body,
@@ -100,7 +100,7 @@ exports.run = async function ({ inputs, appId, emitter }) {
   } catch (err) {
     return {
       success: false,
-      response: err,
+      response: automationUtils.getError(err),
     }
   }
 }

packages/server/src/integrations/utils.ts

@@ -5,6 +5,7 @@ const { DocumentTypes, SEPARATOR } = require("../db/utils")
 const { FieldTypes } = require("../constants")
 
 const DOUBLE_SEPARATOR = `${SEPARATOR}${SEPARATOR}`
+const ROW_ID_REGEX = /^\[.*]$/g
 
 export function isExternalTable(tableId: string) {
   return tableId.includes(DocumentTypes.DATASOURCE)
@@ -32,6 +33,20 @@ export function generateRowIdField(keyProps: any[] = []) {
   return encodeURIComponent(JSON.stringify(keyProps).replace(/"/g, "'"))
 }
 
+export function isRowId(field: any) {
+  return Array.isArray(field) || (typeof field === "string" && field.match(ROW_ID_REGEX) != null)
+}
+
+export function convertRowId(field: any) {
+  if (Array.isArray(field)) {
+    return field[0]
+  }
+  if (typeof field === "string" && field.match(ROW_ID_REGEX) != null) {
+    return field.substring(1, field.length - 1)
+  }
+  return field
+}
+
 // should always return an array
 export function breakRowIdField(_id: string | { _id: string }): any[] {
   if (!_id) {

packages/server/src/utilities/fileSystem/index.js

@@ -106,64 +106,86 @@ exports.apiFileReturn = contents => {
 }
 
 /**
- * Takes a copy of the database state for an app to the object store.
- * @param {string} appId The ID of the app which is to be backed up.
- * @param {string} backupName The name of the backup located in the object store.
- * @return The backup has been completed when this promise completes and returns a file stream
- * to the temporary backup file (to return via API if required).
+ * Local utility to back up the database state for an app, excluding global user
+ * data or user relationships.
+ * @param {string} appId The app to backup
+ * @param {object} config Config to send to export DB
+ * @returns {*} either a string or a stream of the backup
  */
-exports.performBackup = async (appId, backupName) => {
-  return exports.exportDB(appId, {
-    exportName: backupName,
+const backupAppData = async (appId, config) => {
+  return await exports.exportDB(appId, {
+    ...config,
     filter: doc =>
       !(
         doc._id.includes(USER_METDATA_PREFIX) ||
-        doc.includes(LINK_USER_METADATA_PREFIX)
+        doc._id.includes(LINK_USER_METADATA_PREFIX)
       ),
   })
 }
 
 /**
- * exports a DB to either file or a variable (memory).
- * @param {string} dbName the DB which is to be exported.
- * @param {string} exportName optional - the file name to export to, if not in memory.
- * @param {function} filter optional - a filter function to clear out any un-wanted docs.
- * @return Either the file stream or the variable (if no export name provided).
+ * Takes a copy of the database state for an app to the object store.
+ * @param {string} appId The ID of the app which is to be backed up.
+ * @param {string} backupName The name of the backup located in the object store.
+ * @return {*} a readable stream to the completed backup file
  */
-exports.exportDB = async (
-  dbName,
-  { exportName, filter } = { exportName: undefined, filter: undefined }
-) => {
-  let stream,
-    appString = "",
-    path = null
-  if (exportName) {
-    path = join(budibaseTempDir(), exportName)
-    stream = fs.createWriteStream(path)
-  } else {
-    stream = new MemoryStream()
-    stream.on("data", chunk => {
-      appString += chunk.toString()
-    })
-  }
-  // perform couch dump
+exports.performBackup = async (appId, backupName) => {
+  return await backupAppData(appId, { exportName: backupName })
+}
+
+/**
+ * Streams a backup of the database state for an app
+ * @param {string} appId The ID of the app which is to be backed up.
+ * @returns {*} a readable stream of the backup which is written in real time
+ */
+exports.streamBackup = async appId => {
+  return await backupAppData(appId, { stream: true })
+}
+
+/**
+ * Exports a DB to either file or a variable (memory).
+ * @param {string} dbName the DB which is to be exported.
+ * @param {string} exportName optional - provide a filename to write the backup to a file
+ * @param {boolean} stream optional - whether to perform a full backup
+ * @param {function} filter optional - a filter function to clear out any un-wanted docs.
+ * @return {*} either a readable stream or a string
+ */
+exports.exportDB = async (dbName, { stream, filter, exportName } = {}) => {
   const instanceDb = new CouchDB(dbName)
-  await instanceDb.dump(stream, {
-    filter,
+
+  // Stream the dump if required
+  if (stream) {
+    const memStream = new MemoryStream()
+    instanceDb.dump(memStream, { filter })
+    return memStream
+  }
+
+  // Write the dump to file if required
+  if (exportName) {
+    const path = join(budibaseTempDir(), exportName)
+    const writeStream = fs.createWriteStream(path)
+    await instanceDb.dump(writeStream, { filter })
+
+    // Upload the dump to the object store if self hosted
+    if (env.SELF_HOSTED) {
+      await streamUpload(
+        ObjectStoreBuckets.BACKUPS,
+        join(dbName, exportName),
+        fs.createReadStream(path)
+      )
+    }
+
+    return fs.createReadStream(path)
+  }
+
+  // Stringify the dump in memory if required
+  const memStream = new MemoryStream()
+  let appString = ""
+  memStream.on("data", chunk => {
+    appString += chunk.toString()
   })
-  // just in memory, return the final string
-  if (!exportName) {
-    return appString
-  }
-  // write the file to the object store
-  if (env.SELF_HOSTED) {
-    await streamUpload(
-      ObjectStoreBuckets.BACKUPS,
-      join(dbName, exportName),
-      fs.createReadStream(path)
-    )
-  }
-  return fs.createReadStream(path)
+  await instanceDb.dump(memStream, { filter })
+  return appString
 }
 
 /**

packages/string-templates/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@budibase/string-templates",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "description": "Handlebars wrapper for Budibase templating.",
   "main": "src/index.cjs",
   "module": "dist/bundle.mjs",

packages/worker/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@budibase/worker",
   "email": "hi@budibase.com",
-  "version": "0.9.169-alpha.1",
+  "version": "0.9.169-alpha.11",
   "description": "Budibase background service",
   "main": "src/index.js",
   "repository": {
@@ -29,8 +29,8 @@
   "author": "Budibase",
   "license": "AGPL-3.0-or-later",
   "dependencies": {
-    "@budibase/auth": "^0.9.169-alpha.1",
-    "@budibase/string-templates": "^0.9.169-alpha.1",
+    "@budibase/auth": "^0.9.169-alpha.11",
+    "@budibase/string-templates": "^0.9.169-alpha.11",
     "@koa/router": "^8.0.0",
     "@sentry/node": "^6.0.0",
     "@techpass/passport-openidconnect": "^0.3.0",