Novell Filr 1.1: Test Report—Net Folder Synchronization (OES)

White Paper - September 2014

Novell testers created a test bed consisting of an OES server and three virtual appliances (VAs), as illustrated below.

1.0 Executive Summary

When you deploy Filr, the metadata for targeted folders and files must be synchronized with Filr for users to see them in their assigned Net Folders.

Subsequent to your initial deployment and synchronization, you must ensure that file and directory metadata is kept current through regularly scheduled synchronizations.

This white paper summarizes Novell synchronization test results to assist you in assessing the time it could take your organization to do the following:

Initially synchronize file and folder metadata on NSS volumes with Net Folders on Filr.
Keep your Net Folder metadata current with your file servers.

2.0 Methodology

2.1 Synchronization-Only Focus

During the test, only the Filr initial synchronization processes ran on the ESXi box. Content Indexing and Just-in-time synchronization (JITS) were turned off.

2.2 One Physical Box

As illustrated above, the Filr VAs and the OES server were hosted on a single VMware ESXi server.

2.3 No Network Traffic Involved

All communication was local to the VM host server. No external network traffic was generated.

2.4 Multithreading Default Not Changed

The Net Folder synchronization process is multithreaded and the synchronization tests were run using the default synchronization settings, which are:

Up to five Net Folders synchronizing simultaneously

For example, a system with five or more Net Folders will work on synchronizing up to five NetFolders simultaneously until all of the Net Folders have been synchronized.

The testbed used for this whitepaper had 45 Net Folders, so five synchronized at the same time until all were completed.
Four threads devoted to each synchronization process

Multithreading of Net Folder synchronization processes has contributed to the substantial synchronization improvements in Filr 1.1.

The system administration utility lets you configure the number of threads for each sync process. This number cannot exceed three times the number of processors configured for the Filr appliance.

For the testbed Filr appliance (which has four processors configured), this means that the synchronization process could have had as many as 12 threads devoted to each Net Folder synchronization (four CPUs * three threads). This would result in a total of 60 active threads servicing sync processes (5 Net Folders at a time * 12 threads = 60).

However, the thread setting was not modified, so the 45 Net Folders were synchronized using four threads each, resulting in a total of 20 threads. In the testbed, the default settings resulted in quite good performance and did not need optimization.

For more information, see Net Folder Configuration in the Novell Filr 1.0 Appliance Configuration Guide.

3.0 Results

3.1 First Synchronization

First synchronizations establish the Net Folder file and folder metadata. They take longer than subsequent synchronizations, which track changes and update changed metadata.

Directories Synchronized	940,031
Files Synchronized	10,674,262
Total Objects Synchronized	11,614,293
Net Folders Populated	45
Total Time	24 hr. 59 sec.
Synchronization Rate	134.33 objects per second

3.2 Subsequent Synchronization

Subsequent synchronizations occur when files or directories have changed. If nothing has changed, no synchronization is required.

In this case, none of the files had changed.

Directories Updated	940,031
Files Updated	10,674,262
Total Objects Updated	11,614,293
Net Folders Updated	45
Total Time	9 hr. 6 min. 21 sec.
Synchronization Rate	354.3 objects per second

4.0 Observations

4.1 More Files and Directories Take Longer to Sync

The larger and more complex a data structure is, the longer the synchronization takes.

4.2 Synchronization Is linear

The number of directories and files and the complexity of the directory structure affect how long synchronizations take. The synchronization rate across any given system is generally constant.

For example, the Novell first-synchronization test took roughly one day to synchronize 10,000,000 files. Because we know that synchronization is linear, we know that a first-synchronization of 70,000,000 files would take about 7 days or one week.

Organizations can estimate how long it will take to synchronize their Net Folders by synchronizing a small, representative subset of their data (for example 50,000 files) and extrapolating from the results obtained, how long it will take to synchronize all of their Net Folders.

4.3 Changed Vs. Unchanged Data

As you plan Net Folder management and synchronization schedules, keep in mind that subsequent synchronizations of folders and files take slightly longer than one third the time that first synchronizations do, depending on the number of files and folders that have changed.

When synchronizations are triggered, the files and folders that have changed take longer to synchronize than those that haven’t changed.

4.4 Overlapping Synchronizations

It is possible for subsequent synchronizations to take longer than the synchronization-check interval. When this happens, a new synchronization is scheduled before the previous synchronization has finished. As soon as the previous synchronization finishes, the new synchronization starts.

In such cases, the Filr deployment gets into a state where its resources are always tied up with synchronizations.

It is a good idea to avoid overlapping synchronizations by measuring how long a synchronization cycle requires, and then setting the synchronization-check interval to a longer period than that.

4.5 System Resources Affect Synchronization Times

Test results showed that adding processors and RAM improves synchronization times.

5.0 More Information

See the following sections in the Filr 1.1 documentation:

6.0 Legal Notices

Copyright © 2014 Novell, Inc. All rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the publisher. For Novell trademarks, see the Novell Trademark and Service Mark list. All third-party trademarks are the property of their respective owners.