INDEX
| Raweb 2003 / Project-Team : reso
|
HELP INDEX |
|
Key words: .
Data storage in a cluster environment requires dedicated systems that are able to sustain high bandwidth needs and serve many concurrent clients. Several projects have already been proposed to address this issue. PVFS, GPFS or Lustre provide parallel file systems whose scalability is ensured by data stripping and workload sharing across several servers.
We study the link between clients and these systems in order to maximize the underlying network utilization. Indeed cluster nodes are connected through a high bandwidth low latency network such as Myrinet, whose features lead us to the idea of using them for data storage. ORFA (Optimized Remote File-system Access) [46] was developed on Myrinet networks to provide an efficient access to remote data. The user-level implementation showed that file transfers may saturate the physical link [47] . The need to cache metadata on the client's side leads to the idea of porting ORFA into the Linux kernel. Besides, the use of ORFA-like techniques in parallel filesystems should enhance their performance to make the most out of the underlying network.
This work also showed that the now well-known memory registration model that is used on asynchronous network interface such as Myrinet does not fit file system implementation needs. We are currently preparing a collaboration with Myricom to work on a new interface that will fit both filesystem and usual communication that MPI applications use.
Key words: Parallel networking sub-system, data locality, SMP machine, performance, robustness.
We propose a new networking subsystem architecture built around a packet classifier executing in the Network Interface Controller (NIC). By classifying packets in the NIC, we believe that significant performance, scalability, and robustness gains can be achieved on shared-memory multiprocessor Internet servers. To show the feasibility and the benefits of the approach, we developed the KNET software prototype (consisting in extensions to the Linux kernel and modifications to the Myrinet NIC firmware and driver) and ran a series of experiments.
KNET's objectives are to parallelize packet processing in the operating system while maximizing data locality in the processor caches and eliminating the Receive Livelock effect that can severely affect the operating system's robustness. KNET uses per-processor network threads to achieve parallelism, packet demultiplexing in the Network Interface Controller to maximize connection data locality and ensure robustness [35] . KNET exhibits up to 35% improvement in throughput on a 4-way machine.
|
|
|