What Deduplication does CTERA Support

CTERA supports multi-level deduplication on both the CTERA portal and edge filers, enabling major saving in storage capacity required for the backend object and frontend cache storage as well as reducing the network bandwidth required for edge to cloud synchronization.

Deduplication is set by default both at the portal and edge filer levels but can be tuned based on your security requirements and the type of data you have.

Deduplication on CTERA Portals

CTERA Portal splits each stored file into blocks and performs block-level deduplication automatically for all data stored on the backend object storage.

Portal deduplication is also used for optimizing the traffic between edge filers and the portal, guaranteeing that only unique blocks are sent when syncing data between the local cache and the portal global file system.

All users in a team portal share, by default, a single folder group, enabling cooperative global deduplication between all members of the group. Administrators have the option to limit the scope of the deduplication to a user level or up to a cloud folder group level, based on security and scale requirements.

It is possible to tune deduplication performance by changing the block size settings. Increasing the average block size setting or the fixed block size setting causes the files to be split into larger chunks before storage, and results in increased read/write throughput at the cost of a reduced deduplication ratio. For example, with the default 4MB fixed block size, a file of 3MB will be uploaded as a single 3MB block and a file of 5MB will be uploaded as two blocks, 4MB and 1MB.

Increased block size is useful for workloads that require high performance, as well as for those that do not gain greatly from deduplication. For example, where the stored files consist mostly of videos, images, and music files that are not frequently modified. For example, you may want to change the defaults to use fixed block size if many of the folder groups include large files, such as media files, where deduplication is not common.

Decreasing the average block size can result in better deduplication, since the portal can better identify finer-grained duplicate data.

Deduplication on Edge Filers

CTERA edge filers support file-level deduplication that reduces physical cache storage and improves performance when accessing stub files that already exists as another file on the edge filer.

Files with the same content are linked to the same on-disk data, instead of taking twice the disk space. A deduped file, upon being modified, is populated with the underlying data.

Files are deduped:

• During a copy and paste operation in Windows File Explorer or Mac Finder.
• After a file is modified locally, during the upload to the portal, the file will be deduplicated with a local copy of the data, if one exists.
• If a local copy exists before a download starts, deduplication is immediate and the file is not downloaded, saving the network traffic.