Table of Contents
NetDocuments ndMirror is a paid add-in service connected to a NetDocuments repository. ndMirror is a new generation of the original NetDocuments Local Document Service based on the Microsoft .NET Framework using modern engineering practices and technology stack. The diagram below illustrates the NetDocuments ndMirror workflow.
The NetDocuments ndMirror workflow
The customer file system stores documents in their native format, along with an SQL database that contains the following document profile data:
- Document name
- File format
- Creator and the date of creation
- Last modified user and date
- Document ACL
- Other configuration attributes defined by the customer
Documents use a scheme defined by the customer that dynamically stores documents in a folder structure based on the document profile attributes. You can also integrate ndMirror with an external searching application to provide a way of locating documents within the local storage system. You can also gather analytics given all document metadata is stored locally.
This section highlights the key features of the ndMirror application.
An intuitive and self-guiding installation program that configures and sets up the application. It guides a user through the setup process, notifies about any incompatibilities, and prompts for the required software.
Calls to the NetDocuments REST API are authenticated with the help of the OAuth 2.0 protocol. For more information about authentication, see NetDocuments Authentication.
ndMirror takes advantage of multithreading if your server has more than one virtual logical CPU. For example, you can run four times the number of cores so a four-core machine yields up to 16 threads. The advantage of multithreading is that it significantly boosts the performance of the application.
Multithreading is highly recommended. When the synchronization service processes a block of documents and profiles to mirror, it uses some threads and makes a separate call to the REST API on each of those threads. When one job finishes, then the next job starts running if there is any, and this is a continuous process.
A Microsoft SQL Server database stores NetDocuments connection configuration parameters, synchronization status, document profile metadata, ACL, users and groups, lookup tables, subsequent updates, and logs.
Columns within the table include metadata values. This provides more flexibility and sophistication with network and server configurations. Additionally, third-party search vendors have more capabilities and the ability to crawl customer data and create the index. For more information, see SQL Database.
ndMirror and SQL Server Maintenance
ndMirror requires an instance of SQL Server to store the various data it collects in addition to the documents it stores on a disk. While ndMirror manages the logs for sync sessions and the details about the data it downloads, the SQL Server has its own transaction logs that may need maintenance depending on the chosen recovery model.
It is important to decide carefully which strategy to use for backing up and managing the transaction logs (if enabled). Without this strategy, a database failure may require downloading all the data again or may result in databases that are utilizing too much disk space and performing poorly due to the lack of regular maintenance.
There are many ways to deploy and maintain Microsoft SQL Server. NetDocuments recommends an implementation strategy that follows the organization’s existing business rules as well as Microsoft best practices.
High Availability Solutions
NetDocuments recommends ensuring that your data is protected. This requires deploying Microsoft SQL Server in a highly available environment. The specific deployment decision is the responsibility of the organization. We recommend the following Microsoft best practices. Information on the deployment solutions that are available can be found here: https://msdn.microsoft.com/en-us/library/ms190202(v=sql.120).aspx.
Backups and Maintenance
In addition to the deployment methodology and architecture of Microsoft SQL Server, the individual databases must also be maintained on a regular basis. Database and transaction log backups should be performed at regular intervals in accordance with the organization’s disaster recovery plan objectives.
In addition, regular database index maintenance is required for optimal performance. NetDocuments recommends utilizing backup and maintenance strategies that comply with those objectives. If the organization does not currently have a customized set of backup and maintenance plans, NetDocuments recommends starting with industry best practice examples. A few examples can be found here:
- Ola Hallengren’s SQL Server Backup, Integrity Check, and Index and Statistics Maintenance
- Paul S. Randal’s SQL Server Top Tips for Effective Database Maintenance
Users and Groups List
The SQL database stores the list of users and groups and respective cabinets. This data provides third-party search vendors with the ability to resolve security for users that access documents through their external search. As an administrator, you can specify how often to update the list by clicking the Sync Frequency subtab of the Settings tab on the Web Administrative Dashboard.
Status and Event Reporting
As an administrator, you can view the status of ndMirror activity, such as the most recent date and the time of sync event from the ndMirror Web Administrative Dashboard. You can also view daily email summary reports.
Skipped Documents Retry
Synchronization jobs have a recovery mechanism. A document fails after ten retries. It is possible to re-queue or delete the failed job on the Maintenance page of the Web Administrative Dashboard. There is also a cleanup mechanism to remove outdated jobs. For example, the document’s failed job is deleted automatically if a document attempts to sync with further synchronization sessions.
The synchronized data is stored without any protection or encryption on the file system. This is the responsibility of the ndMirror administrator to implement the proper security access and encryption level to the folders with the synchronized data. The ndMirror application uses the SQL Server security model to access the database. See Authentication in SQL Server. The database should be protected with standard SQL Server protection mechanisms in conjunction with the organizations' security best practices. NetDocuments recommends using Windows integrated security as your authentication choice as SQL Server login names and passwords are passed unencrypted across the network. If a user selects the SQL login access (username/password), ndMirror stores login and password encrypted.
The access to the ndMirror Administrative Dashboard Web application is protected with Windows/Domain authorization.
ndMirror runs on a port that is user-selected. Installer opens this port in Windows Firewall. ndMirror maintains the access list of users and groups that can use the application. To edit the access list of users and groups, do one of the following:
- In the Web Administrative Dashboard dialog box of the installer, select the Find link to add users and groups. The installer creates the ndMirrorAdministrators group automatically. You can remove the ndMirrorAdministrators group.
- In the ndMirror Web Administrative Dashboard, select the Settings tab, and then the Administrator Access subtab to edit the access list. A user can review the current access list and add or remove users or groups.
ndMirror synchronization service works on behalf of an administrator user and uses OAuth protocol to authorize the access. Communication with NetDocuments Cloud DMS is protected with an SSL communication protocol. ndMirror always initiates outbound connections to NetDocuments Cloud DMS and it does not require opening an inbound port.
Federated identity allows users to authenticate to NetDocuments using your organization’s identity provider allowing the organization to control the authentication requirements. SAML 2.0 protocol is used for implementing this linking.
While installing ndMirror, in some environments, a user may not be able to see the authorization code on the same page, as some authorization methods cannot display it. To display the code on the separate page, a user needs to click the link in the Note of the Adding credentials to US dialog box, copy, and then paste the code back into the corresponding field. For more information, see Post-installation Configuration Steps.
ND Filing Locations
ndMirror only downloads files or metadata. You can configure ndMirror to create folders on the on-premises server based on profile attributes, for example, client, matter, doc. type. Currently, ndMirror excludes ndfld (folders), ndsq (Saved Searches), and ndfit (filters), ndws (Workspaces), and ndcs (Collaboration Spaces).
ndMirror consists of three logical sections:
- Installer setup – Installs the application, validates prerequisites, and configures the required components.
- Synchronization service – Polls NetDocuments Data Centers and customer’s repository for changes to documents and interrelated metadata to retrieve updates and store the data at a local or remote file system.
- Web Administrative Dashboard – Configures connection to NetDocuments and monitors synchronization service activity.