Directory quotas let you cap the storage size and file count of OSS-HDFS directories, preventing runaway growth and unexpected charges.
OSS-HDFS supports two quota types:
Capacity quota: the maximum total storage a directory can use (including all objects and subdirectories). When the quota is exceeded, object and subdirectory creation in that directory fails.
Quantity quota: the maximum number of objects and subdirectories a directory can contain. When the quota is exceeded, object and subdirectory creation in that directory fails.
Limitations
Quotas apply to a directory as a whole. Quotas based on individual object storage classes are not supported.
Usage notes
Directory usage statistics are not updated in real time. When a directory contains a large number of objects or has high capacity utilization, actual usage may exceed the configured quota.
Prerequisites
Before you begin, ensure that you have:
An OSS bucket with OSS-HDFS enabled
An AccessKey pair (AccessKey ID and AccessKey secret) for your Alibaba Cloud account or a RAM user with the required permissions
The JindoFS SDK installed on your machine
Configure directory quotas
Step 1: Download and install the JindoFS SDK
Download the JindoFS SDK installation package and extract it.
Step 2: Set up credentials and environment
Go to the
bindirectory of the extracted JindoFS SDK package. The following example uses the Linux package:cd jindofs-sdk-x.x.x-linux/bin/Replace
jindofs-sdk-x.x.x-linuxwith your actual package version name.In the
bindirectory, create a configuration file namedjindofs.cfgand add your AccessKey pair:[client] fs.oss.accessKeyId = <your-access-key-id> fs.oss.accessKeySecret = <your-access-key-secret>Placeholder Description <your-access-key-id>Your AccessKey ID <your-access-key-secret>Your AccessKey secret Set the
JINDOSDK_CONF_DIRenvironment variable to the absolute path of thejindofs.cfgconfiguration file:export JINDOSDK_CONF_DIR=<JINDOSDK_CONF_DIR>
Step 3: Enable the directory quota feature
Run the following command to enable the directory quota feature for your bucket:
./jindofs admin -putConfig -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/ -conf namespace.directory.quota.enable=true| Placeholder | Description | Example |
|---|---|---|
<bucket> | Name of the bucket with OSS-HDFS enabled | my-bucket |
<oss-hdfs-endpoint> | OSS-HDFS endpoint for your region | cn-shanghai.oss-dls.aliyuncs.com |
To verify that the feature is enabled:
./jindofs admin -getConfig -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/ -name namespace.directory.quota.enableStep 4: Set quotas on a directory
Replace <bucket>, <oss-hdfs-endpoint>, and <path> with your actual values in all commands below.
Set a quantity quota (maximum number of objects and subdirectories):
./jindofs admin -setQuota -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/<path> -q 1000The -q 1000 flag sets the limit to 1,000 objects and subdirectories combined.
Set a capacity quota (maximum storage size, in bytes):
./jindofs admin -setSpaceQuota -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/<path> -q 1000The -q 1000 flag sets the limit to 1,000 bytes.
Verify the configured quotas:
./jindofs fs -count -q -v -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/<path>Step 5: Remove quotas (optional)
If your business requirements change or the storage capacity of a directory exceeds your initial plan, remove the quotas from a directory.
Remove the quantity quota:
./jindofs admin -clearQuota -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/<path>Remove the capacity quota:
./jindofs admin -clearSpaceQuota -dlsUri oss://<bucket>.<oss-hdfs-endpoint>/<path>