You just created a dataset. You now need to upload some files into it.
For now, access and secret keys are restricted to a single dataset. We plan to allow the creation of project and organization-based keys in the near future.
Datasets, under-the-hook, S3-compatible buckets. You can use all the tools of the S3 environment. In order to read, write or delete files to your datasets, you need to get the dataset access and secret keys.
- Login/sign up to your csquare account.
- Select the project and got to the dataset page.
- Copy your access and secret key.
- Select the project you want to work with.
- Go to the
Datasetspage using the sidebar.
- Click on the ID of the dataset you want to work with to access the file explorer.
- Drag-and-drop the files to upload in the explorer.
You just updated your dataset. Well done!
s3cmd command line interface#
You can also use the famous
s3cmd CLI to upload files to your datasets. You can install
s3cmd from the (official s3cmd website)[https://s3tools.org/s3cmd].
You can check the installation status of s3cmd by running:
which should output:
Once you have a working CLI, you can configure
s3cmd by running:
and answer the interactive questions:
You can then use the default configuration values by hitting
Enter as many times as required.
Do not forget to save your configuration.
You can now test the connection by listing the associated buckets:
All the available commands are accessible by running
s3cmd --help or on the
If you do not want to save your
s3cmd configuration or want to use
s3cmdn without the
--configure flag, you can pass the following options.
s5cmd command line interface#
s5cmd is a more powerful than
s3cmd and supports parallelization of the uploads and has in general much better performances than most of the other S3 CLI clients.
However, usage of
s5cmd is a bit more complex than
s3cmd because it depends on the AWS CLI configuration files.
- Install the AWS CLI version 2 (no AWS account is required), then check the AWS CLI installation via the following command:
- Install s5cmd: follow the installation instructions, ensure that
s5cmdis properly accessible with:
Configure your dataset access and secret keys. In this example, we are using the dataset
my-dataset with its access key
C2-2zJBcUJTG0GIHbvpsm9xe and secret key
s5cmd using the
AWS_PROFILE environment variable and
Refer to the
s5cmd documentation for more information.