National Data Storage (NDS) project is intended to build a nation-wide, distributed storage system, that provides high efficiency of data access and I/O operations as well as high level of security, reliability and data durability.
NDS is a research project that aims in development of the software that implements the distributed storage system services. The services are going to be deployed in the environment of the Polish Optical Internet PIONIER network that spans scientific and educational institutions in Poland and in Metropolitan Area Networks (MAN). The main application of the system is the backup/archival data storage service for academic, educational, local government and other public institutions.

Services offered by NDS project:

Backup/Archive service,

Virtual system access through protocols: SCP, HTTPs and GridFTP.

Backup/Archive service offers a backup/archival functionality to end-users. The service supports on-the-fly replication, encryption of the data (before they leave the user system), integrity control as well as typical backup/archive scenarios such as full and incremental backup and data archival.

Virtual filesystem service provides an easy interface to store and retrieve data. The service implements the data replication, that is transparent to end-users. Users have illusion of interacting with the remote SCP, HTTP or GridFTP site, containing the virtual filesystem. This logical view hides the complexity of the physical data repository and details of data processing happening in the system such as physical data replication.

Replication. Users data may be stored in multiple replicas located in geographically-distant NDS centers. In case of failure of default storage node, the data can be retrieved from any existing replicas. This feature is supported by both services provided by NDS.

NDS architecture is discussed in more details here.

support for geographical distribution of services,

high persistency of data & meta-data,

automatic data replication across system nodes,

meta-data and data recovery mechanisms,

scalability in many dimensions: storage space, number of data objects, volume of client traffic.

Unique features of NDS:

Support for both synchronous and asynchronous data replication (!),

Semi-synchronous replication of meta-data (!),

Separation of logical namespaces of users, which improves scalability & security,

User can access the data using typical protocols and tools such as SCP, WebDAV or GridFTP clients,

Data replication is made transparently and automatically, not affecting the system's interface to users,

Detailed accounting support: user vs system and system node vs system node.

The results of the NDS project are going to be put into practice as the National Data Storage service. Multiple access and service points spread across Poland will provide high efficiency of data I/O, data durability and high availability of the services across the country.