22-04-2011, 10:57 AM
[attachment=12606]
1.Introduction :-
Reliable and efficient data storage is a major concern in the modern computer industry. This type of storage is mainly provided by magnetic and optical devices, the most common being the traditional magnetic disks. However, since processor performance improves at a higher rate than disk performance, the cost of a disk access (measured in processor cycles) continues to increase with time. As a result, several applications whose effective execution depends on low latency access to their files, are going to suffer an increasingly high performance penalty if they store their data on traditional magnetic disks. Such applications include transaction-based systems, visualization systems, web servers, web proxies, compilers, etc.
Several file systems attempt to speedup file accesses by using a portion of the file server's main memory as a disk cache. Unfortunately, such caches can not be larger than a single workstation's main memory. To overcome this memory size limitation, research file systems use all client and server caches in a NOW as a single cooperative cache. Unfortunately, such file systems are not widespread yet, and their principles have not been incorporated into commercial operating systems.
Here a new device, the Network RamDisk is introduced, which attacks the high latency problem in a very convenient manner. A Network RamDisk is a block device that consists of the all idle main memories in a Network of Workstations. It behaves like any other disk, allowing the creation of files and file systems on top of it. However, since it is implemented in main memory, it provides lower latency and higher bandwidth than most traditional magnetic disks.
In this paper, The organize the client and server memories of a NOW into a Network RamDisk is proposed. Idle workstations donate their memory which will be used to store a portion of the Network RamDisk. When an application reads a file, the file system requests a block from the device driver of the Network RamDisk. The driver knows in which workstation's main memory the block resides, asks the workstation for this block, and returns the requested block. Write operations proceed in the same way. Since block accesses involve only memory and interconnection network transfers, they proceed with low latency and high bandwidth. For example, typical disk latency is around 10 ms, while typical network latency is around 1 ms. Modern interconnection networks provide latency as low as a few microseconds. Thus, Network RamDisks may result in significant performance improvements over magnetic disks, especially when application performance depends on latency.
The Network RamDisk, much like network memory file systems exploits network memory to avoid magnetic disk I/O, but unlike these file systems, the Network RamDisk, being a device driver, can be easily incorporated into any environment, without modifying existing operating system sources, or changing file systems. Thus, by using a Network RamDisk, users will be able to exploit all network memory to store their files, without changing their operating system, or their file system.
2.Concept of Network RAM :-
The aggregate main memory of the workstation is called Network RAM.A significant amount of resources unused in the cluster at any given time. 80-90 % of workstation wil be idle in the evening & late afternoon 1/3 of workstation are completely unused, even at the busiest times of the day 70-85 % of the Network RAM in a cluster is ideal.
The main goal of Network RAM is to improve the performance of memory intensive workloads by paging to idle memory over the Network rather than to disk.
Major assumptions for evaluating Network RAM :-
There are four major assumptions that people make when evaluating Network RAM:
1. Disks are slow.
2. Networks are fast.
3. Most machines have free memory.
4. Software overhead is low.
Possible Network Ram Implementations :-
1. Explicit Program Management :-
This implementation would require user code to explicity move data back and forth from either the disk or the Network.This idea requires substantial code modification.
2. User-Level :-
This solution requires that the user modify their code to use a new malloc which allocates from Network RAM backed memory. This is an easier modification than explicit program management, but can still require source code.It has the overhead of handling interrupt on page faults and managing the page tables through system calls.
3. User Level Pager :-
This solution is possible in such operating system, which allows a user level program to handle moving pages to and from second-level storage.
4. Device Driver :-
Network RamDisk is used as a Device Driver. It is a block device that consists of the all idle main memories in a Network of Workstations. It behaves like any other disk, allowing the creation of files and file systems on top of it. It provides lower latency and higher bandwidth than most traditional magnetic disks. This solution has the advantage that it requires no code modification or kernel source modification. It has disadvantage that OS pages can send over the network, making reliability more important.
5. Modified Kernel :-
This solution should give the higest performance without modifying user program because it requires no extra interrupts or context switches. However, changes to kernels are not portable between architectures.
6. Network Interface :-
This solution requires replacing the memory controller with some of custom memory / network chip, making this solution the most un-portable. This solution can have a low overhead than a modified kernel because it can operate on cache lines rather than entire pages, making the data transferred smaller.
In this paper, Device Driver implementation of Network Ram is discussed.
1.Introduction :-
Reliable and efficient data storage is a major concern in the modern computer industry. This type of storage is mainly provided by magnetic and optical devices, the most common being the traditional magnetic disks. However, since processor performance improves at a higher rate than disk performance, the cost of a disk access (measured in processor cycles) continues to increase with time. As a result, several applications whose effective execution depends on low latency access to their files, are going to suffer an increasingly high performance penalty if they store their data on traditional magnetic disks. Such applications include transaction-based systems, visualization systems, web servers, web proxies, compilers, etc.
Several file systems attempt to speedup file accesses by using a portion of the file server's main memory as a disk cache. Unfortunately, such caches can not be larger than a single workstation's main memory. To overcome this memory size limitation, research file systems use all client and server caches in a NOW as a single cooperative cache. Unfortunately, such file systems are not widespread yet, and their principles have not been incorporated into commercial operating systems.
Here a new device, the Network RamDisk is introduced, which attacks the high latency problem in a very convenient manner. A Network RamDisk is a block device that consists of the all idle main memories in a Network of Workstations. It behaves like any other disk, allowing the creation of files and file systems on top of it. However, since it is implemented in main memory, it provides lower latency and higher bandwidth than most traditional magnetic disks.
In this paper, The organize the client and server memories of a NOW into a Network RamDisk is proposed. Idle workstations donate their memory which will be used to store a portion of the Network RamDisk. When an application reads a file, the file system requests a block from the device driver of the Network RamDisk. The driver knows in which workstation's main memory the block resides, asks the workstation for this block, and returns the requested block. Write operations proceed in the same way. Since block accesses involve only memory and interconnection network transfers, they proceed with low latency and high bandwidth. For example, typical disk latency is around 10 ms, while typical network latency is around 1 ms. Modern interconnection networks provide latency as low as a few microseconds. Thus, Network RamDisks may result in significant performance improvements over magnetic disks, especially when application performance depends on latency.
The Network RamDisk, much like network memory file systems exploits network memory to avoid magnetic disk I/O, but unlike these file systems, the Network RamDisk, being a device driver, can be easily incorporated into any environment, without modifying existing operating system sources, or changing file systems. Thus, by using a Network RamDisk, users will be able to exploit all network memory to store their files, without changing their operating system, or their file system.
2.Concept of Network RAM :-
The aggregate main memory of the workstation is called Network RAM.A significant amount of resources unused in the cluster at any given time. 80-90 % of workstation wil be idle in the evening & late afternoon 1/3 of workstation are completely unused, even at the busiest times of the day 70-85 % of the Network RAM in a cluster is ideal.
The main goal of Network RAM is to improve the performance of memory intensive workloads by paging to idle memory over the Network rather than to disk.
Major assumptions for evaluating Network RAM :-
There are four major assumptions that people make when evaluating Network RAM:
1. Disks are slow.
2. Networks are fast.
3. Most machines have free memory.
4. Software overhead is low.
Possible Network Ram Implementations :-
1. Explicit Program Management :-
This implementation would require user code to explicity move data back and forth from either the disk or the Network.This idea requires substantial code modification.
2. User-Level :-
This solution requires that the user modify their code to use a new malloc which allocates from Network RAM backed memory. This is an easier modification than explicit program management, but can still require source code.It has the overhead of handling interrupt on page faults and managing the page tables through system calls.
3. User Level Pager :-
This solution is possible in such operating system, which allows a user level program to handle moving pages to and from second-level storage.
4. Device Driver :-
Network RamDisk is used as a Device Driver. It is a block device that consists of the all idle main memories in a Network of Workstations. It behaves like any other disk, allowing the creation of files and file systems on top of it. It provides lower latency and higher bandwidth than most traditional magnetic disks. This solution has the advantage that it requires no code modification or kernel source modification. It has disadvantage that OS pages can send over the network, making reliability more important.
5. Modified Kernel :-
This solution should give the higest performance without modifying user program because it requires no extra interrupts or context switches. However, changes to kernels are not portable between architectures.
6. Network Interface :-
This solution requires replacing the memory controller with some of custom memory / network chip, making this solution the most un-portable. This solution can have a low overhead than a modified kernel because it can operate on cache lines rather than entire pages, making the data transferred smaller.
In this paper, Device Driver implementation of Network Ram is discussed.
