Latest News

More GridFTP information: new portal, tutorials, success stories

I found tutorial slides and handouts on the Web, under a new GridFTP portal. This should be quite useful to get people started with these tools. Pointers have been added to the GridFtpProtocol topic.

-- SimonLeinen - 28 Jul 2008

DTrace IP Provider integrated into OpenSolaris

(message from 7 Jul 2008, edited to add note about Nevada b93)

DTrace is a mechanism for system-wide instrumentation for performance analysis and debugging that was pioneered in Sun's Solaris operating system. It has since been added to other systems such as BSD (forgot which variant(s)) and Apple's Mac OS X. In the OpenSolaris community, there is a project to add network -specific probes to DTrace. I just learned that the DTrace IP Provider, which is outlined in this "onepager", was integrated into OpenSolaris Nevada four weeks ago. It was released yesterday as part of Nevada b93.

It is interesting to compare the DTrace approach with the web100 system. I think that there is a large overlap in the things that you can do with both, but the interfaces are quite different. Of course, web100 is very specific to TCP, while DTrace is a general tool for observing system behavior.

-- SimonLeinen - 14 Jul 2008

Linux 2.6.26

Another new Linux kernel version has been released today. Notable new features related to network performance are support for some 802.11n high-speed WLAN cards, and a performance improvement in [[CubicTcp][CUBIC TCP]. See the CUBIC topic for the patch, and for some measurements of CUBIC compared with other TCP variants.

-- SimonLeinen - 14 Jul 2008

Dilbert on Network Performance

-- SimonLeinen - 14 Jul 2008

New GridFTP supports SSH for its control channel

The new version 4.2 of the Globus Toolkit was released on July 2. One of its new features is that its GridFTP component can now use SSH for its control channel. This could make it much more attractive for users who just want to haul large amounts of data over LFNs, but who don't necessarily want to run the full Globus infrastructure. Anybody wants to give this a try?

-- SimonLeinen - 04 Jul 2008

ACM Queue: Kode Vicious articles about Latency and Livelock

ACM Queue is a relatively new publication of the Association for Computing Machinery (ACM) oriented to IT practitioners. It includes a section where Kode Vicious, the slightly psychotic alter ego of George V. Neville-Neil, responds to (real?) questions from readers. The column from the March/April 2008 issue is now available online, with two articles relevant for (network) performance: One talks about the often neglected impact of latency on performance even where bandwidth is plentiful, and the other explains a problem called "livelock", which can be seen when servers get persistently overloaded. At SWITCH, I believe we have seen a nice example of this with the RRD MA server that we host for PerfSONAR. Unfortunately, the article about latency has an annoying decimal-point error, which leads to some hand-waving about router-induced delays that is inappropriate - I posted a correction as a reader comment.

RFC 5236 (Reorder Density)

There is a new RFC with an alternative ("improved") metric for packet reordering. It doesn't seem to be an official product of the IPPM WG, and its status is informational (not standards-track), and it doesn't obsolete RFC 4737. I added the reference to the PacketReordering and IPPerformanceMetrics topics.

LSO vs. MDT

There is an interesting discussion on the OpenSolaris networking forum that is still ongoing. The original topic was whether OpenSolaris supported LSO for UDP (apparently it doesn't). The discussion quickly evolved into a debate of the relative merits of LSO - where the adapter knows how to segment transport data units and construct all headers for them - and MDT, which is like a software-only variant of LSO, where the host (operating system) still has to do the segmentation and header construction, but can send all the segments in a single (bus) transaction. I added a few sentences on this trade-off to the LargeSendOffloadLSO topic.

-- SimonLeinen - 04 Jun 2008

bwctl 1.3 release candidate

The first release candidate for a new version 1.3 of bwctl was released yesterday. Changes include more robust scheduling, improved error handling and logging, parallel streams (-P), CSV output, and support for several throughput measurement tools: thrulay, nuttcp, and iperf. Here is the original announcement.

-- SimonLeinen - 22 Apr 2008

FreeBSD 7.0 networking enhancements

Release 7.0 of the FreeBSD operating system was released on 27 February. This versions claims important improvements in the area of network performance and scalability. It joins Linux and Windows (Vista and newer) in implementing TCP buffer auto-tuning, and also adds large-send (LSO) and large-receive offload (LRO) for some adapters. See this ONLamp article for discussions of some of the enhancements.

-- SimonLeinen - 02 Apr 2008

Wireshark news: 1.0 released; First Sharkfest 31 March - 2 April 2008

Wireshark 1.0 was announced on 31 March 2008. For changes from previous versions, see the Wireshark 1.0.0 Release Notes.

The release was made just in time for the first annual SHARKFEST event, which took place on 31 March - 2 April at Foothill College in Los Altos Hills, CA, USA, and which featured a keynote speech by Vint Cerf.

-- SimonLeinen - 02 Apr 2008 (updated)

Vista Service Pack 1 (SP1) Released

The first Service Pack to Microsoft's Windows Vista operating system was published on Microsoft's download site on 18 March. The Service Pack should be available through Windows Update. Reportedly, the kernel of Vista SP1 is based on Windows 2008, while the kernel in the original Vista was based on Windows 2003.

I have skimmed the "Notable Changes" document for performance-relevant changes in networking code. There are several significant additions such as more encryption and random-number generation mechanisms for IPSec, 802.11n (high-speed wireless LAN) support, and support for full TCP offload (TOE) called TCP Chimney.

-- SimonLeinen - 26 Mar 2008

Discussion on High-Performance Networking and SSH in OpenSolaris community

Sun's Solaris operating system is changing to an open-source/community-based development model, so most of the design discussions about the evolution of that system can now be found on the Web and participated in. I'm following the networking community's forum, and found an interesting thread on Improving HPC network performance, specifically SSH/SCP/SFTP perf starting 14 February 2008. The discussion touches topics of TCPBufferAutoTuning, (HPN-) SecureShell (SSH), and other improvements that would help Bulk File Transfers.

Unfortunately, the discussion stopped short of somebody volunteering to actually start work on these improvements in the form of an OpenSolaris project...

-- SimonLeinen - 07 Mar 2008

Amazon Web Services (AWS) and high-performance networking

It is well known that Amazon uses a large computing and network infrastructure to power its bookstores and other e-business activities, Over the past few years, they have also diversified by commercially offering access to this infrastructure to third parties. These services are called "Amazon Web Services (AWS)", and they include a storage service ("S3" for Simple Storage Service), compute services based on virtual machine environments that can be installed on large numbers of nodes ("EC2" for "Elastic Compute Cloud") and others.

According to an item in yesterday's newsletter, they recently added support for Window Scaling and SACK to the TCP configuration of their servers, to improve performance for their many remote users.

-- SimonLeinen - 05 Mar 2008

News Archive:

• OldNews2007
• OldNews2006