[attachment=15224]
[attachment=15225]
[attachment=15226]
[attachment=15227]
[attachment=15228]
Abstract
Use of peer-to-peer technologies have been prevalent in sharing files and media over internet. This project proposes a new cooperative file sharing and storage system that allows users to store, share and find files over a robust and scalable network. This system is based on peer-to-peer technologies and provides a self-managed, decentralized, efficient, load-balanced and distributed way to store and retrieve files on peer nodes. Each node donates storage space to the system and also does routing of client messages based on Pastry[14] location protocol. This system uses replication based on erasure codes for increasing efficiency in providing persistent storage. The system also provides its users with a global hierarchical directory structure to browse for files – like a file-system. We present the system, its features, its architecture design and our progress in this report.
1 Introduction
Peer-to-peer(P2P) can be de_ned a network where sig- ni_cant part of network's functionality is provided by distributed peers, rather than being implemented on centralized server or multiple servers. A peer is a par- ticipant of such peer-to-peer network, which generally is a single program running on multiple hosts. Broadly speaking, P2P network can be classi_ed as either de- centralized or centralized. Decentralized means func- tionality is equally implemented by all or most of the peers in the network. Centralized means that imple- mentation of functionality also uses programs that are not peers, namely centralized servers, that are di_erent from peers and run on relatively small number of hosts. P2P networks have become very popular over last decade { of which _le sharing is the most widley used application. A P2P _le sharing system is a network which facilitates transmission of _les between peers. Such network allows peers to download _les from other participants of network, and also allow to mark a set of _les from their local _le system to be shared. These shared _les are available to other users for download. Today, there are a number of peer-to-peer systems(Gnu-tella[5], Bit-Torrent[2], Freenet[4], and _CS499: Btech project _nal term report. Napster[7]) aimed at creating cooperative _le storage and sharing system. These applications harness idle storage and network resources of volunteer peers to cre- ate an online storage and sharing system { yet these systems are met with challenges of being reliable and scalable. In this project, we propose a decentralized and unmanaged P2P system, which provides an online, fault-tolerant, strongly persistent storage available to multiple users simultaneously. The system consists of nodes connected to the net- work, where each node is identical, and performs identi- cal jobs. Each node can send messages to other nodes, and is responsible for routing messages sent by other nodes. A node contributes to the network by accepting and storing an amount of data from the network into its own drives. Node could also post data to the network, and download data that other nodes have posted. The system's total storage capacity is aggregate of small amount of data that each node allows to store on her local drive. Thus in a network consisting thousands of nodes, this is seemingly limitless space. The nodes together form a self-organizing and decen- tralized overlay network. Inserted _les are replicated over multiple nodes to allow peers to join and leave with high frequency without a_ecting the persistence of stored data. The system uses optimal erasure codes based on Reed-Solomon codes[12] instead of simply du- plicating _les over nodes. We shall show that erasure codes provide a high availability of _les where expected availability of individual node could be as low as ap- prox. 25% of the time. The system uses a routing scheme based on Pastry[14] location protocol to route clients requests to appropriate nodes. Pastry can reli- ably route a client request in number of hops at most logarithmic order in total number of nodes in network.