Storing data with data connections

For this workshop, you need two S3-compatible object storage buckets, such as Ceph, Minio, or AWS S3:

My Storage - Use this bucket for storing your models and data. You can reuse this bucket and its connection for your notebooks and model servers.
Pipelines Artifacts - Use this bucket as storage for your pipeline artifacts. A pipeline artifacts bucket is required when you create a pipeline server. For this workshop, create this bucket to separate it from the first storage bucket for clarity.

You can use your own storage buckets or run a provided script that creates local Minio storage buckets for you.

Also, you must create a data connection to each storage bucket. A data connection is a resource that contains the configuration parameters needed to connect to an object storage bucket.

You have two options for this workshop, depending on whether you want to use your own storage buckets or use a script to create local Minio storage buckets:

If you want to use your own S3-compatible object storage buckets, create data connections to them as described in Creating data connections to your own S3-compatible object storage.
If you want to run a script that installs local Minio storage buckets and creates data connections to them, for the purposes of this workshop, follow the steps in Running a script to install local object storage buckets and create data connections.