How to write your own strapi upload provider

How to write your own strapi upload provider

For the past two years, strapi has been my go-to choice when I needed to design restful APIs for content pre-packaged with a nice UI to perform data entry. Most of my clients' content-oriented sites are build with strapi as the content source and different technologies for data retrieval (Quarkus or Nest.js).

One of the features that strapi comes out of the box with, is static file uploading. Strapi comes prepackaged with a module to allow file uploading to the local disk. If you are deploying in the cloud, however, you will most probably need to upload your static files.

strapi's file upload UI.

Strapi has provided instructions on how to use an external provider to upload files to your web service. And most of those will cover your needs. However, if you need to upload your files to a lesser-known provider, you will have to create a provider on your own.

Building a custom provider

Let's set up a new strapi upload provider. I have used OVH for this one. OVH has data store offerings using the OpenStack API, for which - at the time of this writing -  there is no provider offering support.

First of all, we need to set up a new NodeJS module in our local file system and include this module from our strapi installation as if it was an external module.

Create a new folder named strapi-provider-upload-ovh inside the ./providers folder which should reside in the root of your strapi installation. If it's not there yet, add it yourself. Inside this folder, add a  package.json file:

{
  "name": "strapi-provider-upload-ovh",
  "version": "1.0.0",
  "maintainers": [
    "my-email.domain.com"
  ],
  "description": "ovh upload provider for strapi",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Christos Sotiriou",
  "license": "ISC",
  "dependencies": {
    "pkgcloud": "^2.2.0",
    "streamifier": "^0.1.1"
  }
}

pkgcloud is a multi-provider compatible library that allows common operations such as retrieving storage files and metadata and manipulating other types of components such as load balancers. It works nicely with the OpenStack API. We will need only its block storage capabilities for now. As for streamifier, we will need it shortly.

The first thing we need to do is to setup the configuration variables for our extension. strapi passes those variables to your extension while calling its methods automatically.

Inside the config/plugins.js add an entry for the upload plugin. Like this:

module.exports = ({ env }) => {
  return {
    upload: {
      provider: 'ovh',
      providerOptions: {
        keystoneAuthVersion: 'v3',
        provider: 'openstack', // required
        username: env('STORAGE_USERNAME','user-id'),
        password: env('STORAGE_PASSWORD', '<<storage-password>>'),
        region: env('STORAGE_REGION', 'DE'),
        domainId: env('STORAGE_DOMAIN_ID', 'default'),
        domainName: env('STORAGE_TENANT_NAME','tenant_name'),
        authUrl: env('STORAGE_AUTH_URL', 'https://auth.cloud.ovh.net/'),
        defaultContainerName: env('STORAGE_CONTAINER_NAME', '<<storage container name>>'),
        publicUrlPrefix: env('STORAGE_PUBLIC_URL_PREFIX', 'your public files URL')
      },
    },
  }
};
cc

The providerOptions of each upload plugin are different. The provider name should be the name of your provider - it's best to have it distinct, like local, aws, or - in my case - ovh . I have filled the provider options with information obtained by my OVH provider. Depending on your use case, that info will change. The basic thing to understand here is that those options will be passed to our plugin as they are declared here - so we will use them as we see fit.

Writing the plugin code

Inside ./providers/strapi-provider-upload-ovh/index.js we need to put the code that our plugin will use. Strapi will use a convention to call this file, passing the configuration we set up in the previous step as an argument.

For file upload providers, we need to declare two methods: upload and delete - those are the methods that will be called by the CMS when uploading and deleting assets. To do this, we need to export a single top method init inside the file, and this method will, in turn, make appropriate setups and return an object with two methods inside; upload and delete

Here is a sample implementation

Perhaps you are wondering about the following line

url: `${providerOptions.publicUrlPrefix}/${result.name}`,

This line concerns specifically the OVH implementation. Actually, when you upload something to an object store, you are using a proper domain name to access those files and not the actual URL of the object storage to your file. You may also have set up a CDN. So, for strapi to know the actual URL to your file, you need to return an object containing the end-user URL to your file. I am doing this here, you may need to use a different method, depending on your provider and implementation.

One other thing to take note of is the way we handle the file streaming. strapi uses node-formidable under the hood, which means that the file passed here is a formidable file object, from which we will use the buffer. We are using streamifier to setup a read stream that will pipe the data as it receives it to our object storage provider.

Now, let's return to our installations root folder, and perform an npm install again. This will take our implementation, and put it inside the node_modules folder of our root strapi project, and make it available to use.

And that's it! You can now run npm run develop on your server, and try to upload a file via the UI. You will see that your file is streamed to your file provider instead of your file system.

Bonus: How to debug strapi?

If you are running the develop script to run your strapi installation, you will have notified that there is no good way to set up a breakpoint and debug things that you are developing, whether that is a custom endpoint, a service, or your plugin.

Fortunately, there is an easy fix for that.

Put a server.js file inside the root of your strapi installation, and fill it with those lines:

const strapi = require('strapi');

strapi(/* {...} */).start();

Now, make a npm run develop to produce the build folder, then stop the server. Using your favorite tool (I use IntelliJ), set up a debugger using the methodology you are using in any NodeJS application, and point to the server.js file as your application's entry point. Your breakpoints will work flawlessly.

This can become handy when testing your file upload provider implementation. Unfortunately, you cannot just put a breakpoint inside the point where you wrote the original code. Instead, if you want to put breakpoints to your module, I need to open the root folder's node_modules, find your module there (it's copied from its original location), and put the breakpoint there. I haven't found a better way of doing this until the time of this writing.

Conclusion

That's it! Not so hard, is it? Strapi has provided a straightforward protocol to adhere to if we want to create custom upload providers. Strapi provides packages that support many upload providers out of the box, but if you find that your upload provider is not used, you can now develop your own solution with ease.