Project Summary

This National Science Foundation (NSF) project investigates optimizing network transfers using introspective modeling of end-systems. The bottleneck for the transferring data at very high speeds often turns out to be the end-system performance. In the absence of definitive knowledge about the workload at the receiving end-system, the sender's transmission rate often overshoots the critical bottleneck rate of the receiver. This typically results in oscillations between the extremes and poor performance. To optimize the performance of the transport protocols and achieve the important flow control functionality, it is important to estimate the receiving end-system effective bottleneck rate. In this project we will use modeling and active analysis of the end-system to estimate this rate.  We will develop queueing network models for representing the different multi-core and multiprocessor end-systems running different types of workloads. We will develop a software tool to be integrated with existing transport protocols. We will carry out experimental analysis for different types of end-systems with different configurations and workloads. We will apply and extend methods that have been proposed to address the limitations of queueing network models for performance analysis of computer systems with bursty workloads and correlated service times.  The software tool will be made available to the research community to analyze and optimize distributed applications and systems. The research project will provide a framework to train graduate and undergraduate students in both analytical and experimental methods, and develop knowledge and intuition about next generation computer systems and distributed applications.

Project Title and Duration: [0917315/08-004661] Estimating the End-system Network I/O Bottleneck Rate to Optimize Transport Layer Performance. September 2009 - August 2012. Funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5).

People

Vishal Ahuja
Rennie Archibald
Amitabha Banerjee
Matt Farrens
Dipak Ghosal
Giuseppe Serazzi

[Publications] [Talks] [Tools and Code] [Miscellaneous]

New

1. V. Ahuja,  R. Archibald, A. Banerjee, M. Farrens, and D. Ghosal, "Estimating the Network I/O BottleneActive End-System Analysis to Estimate the Network I/O Bottleneck Rate,"  Workshop on The Influence of I/O on Microprocessor Architecture (IOM-2009) Raleigh, North Carolina, February 15, 2009.

2. A. Banerjee, D. Ghosal, B. Mukherjee, and W. Feng, Algorithms for Integrated Routing and Scheduling for Aggregating Data from Distributed Resources on a Lambda Grid"  IEEE Transaction on Parallel and Distributed Systems, 2008, vol. 19, Issue 1, pp. 24-34.

3. A. Banerjee, D. Ghosal, and B. Mukherjee, Modeling and Analysis to Estimate the End-System Performance Bottleneck Rate for High-Speed Data Transfer, Fifth International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet) 2007, Los Angeles.

4. A. Banerjee, W. Feng, B. Mukherjee, and D. Ghosal, RAPID: An End-System Aware Protocol for Intelligent Data-Transfer over Lambda-Grids, in the Proceedings of the IEEE/ACM International Parallel and Distributed Processing Symposium (IPDPS 2006), Rhode Island, Greece, April 2006.

5. A. Banerjee, W. Feng, B. Mukherjee, and D. Ghosal, "End-system Performance Aware Transport over Optical Circuit-Switched Connections," IEEE INFOCOM High-Speed Networking Workshop: The Terabits Challenge, April 2006.

6. N. Rao, Q. Wu, S. Carter, W. Wing, A. Banerjee, D. Ghosal, and B. Mukherjee, "Control Plane for Advance Bandwidth Scheduling in Ultra High-Speed Networks," IEEE INFOCOM High-Speed Networking Workshop: The Terabits Challenge, April 2006.

7. A. Banerjee, W.-C. Feng, B. Mukherjee, and D. Ghosal, Routing and Scheduling Large File Transfers over Lambda Grids, Third International Workshop on Protocols for Fast Long-Distance Networks (PFLDNet) 2005, February 3,4 2005,Lyon France.