Fulcrum Microsystems Blog

By Gary Lee

In my last post, I introduced the new FlexPipe™ packet-processing pipeline, a high performance packet engine for Fulcrum’s Alta High-Speed Ethernet switch architecture that can provide sustained performance of a billion packets per second while maintaining less than 300ns latency.  In this post, I want to cover the microcode support built into FlexPipe that delivers a new level of flexibility for supporting current and future standards and protocols without sacrificing performance.

Data center standards are constantly evolving, leaving chip makers to either wait to hard-code the protocols in the device, or to use lower performance programmable silicon and count on a software-based enhancement after the chip has been designed into a system.  Neither is ideal. But the support for microcode built into FlexPipe gives future Alta chips the performance of a hard-coded solution with the flexibility of a software solution.

Typically, when one hears the term microcode and programmable, it is assumed that the architecture is a run-to-completion or non-deterministic model.  FlexPipe operates in a deterministic manner, meaning that with any possible microcode implementation, the engine will maintain more than one billion packets per second of throughput performance while maintaining less than 300ns of latency in cut-through mode.

With the use of microcode, we can provide customers with a rich set of flexible features that can be adapted to changing market needs. Our customers can get to market early, before industry groups finalize standards, allowing future-proof system designs. It also allows us to anticipate standards that have not even been thought of yet.   Some of the known standards where the programmability comes into play include RBridge/Trill, Shortest Path Bridging, VEB, VEPA, VEPA+ multi-channel, VN-Tag, Fibre Channel, LISP or other proprietary switching headers for vertical markets.  We also plan to support OpenFlow, and will be able to keep up with this implementation as it evolves.

FlexPipe is the key differentiator for future Alta chips, giving them the performance and flexibility they need to deliver the top-of-rack switch performance necessary for high-port count virtualized networks.

Comments welcome: feedback@fulcrummicro.com

By Gary Lee

A billion packets per second.  That’s the performance of FlexPipe™, the new packet-processing engine that’s part of the Alta high-speed Ethernet switching architecture announced by Fulcrum CTO Uri Cummings at Hot Interconnects last week.

What network switch needs a billion packets per second performance?  Let’s stop first to recognize just what we’re talking about because a billion of anything is hard to visualize.

In months it would equal 82 million years (back when dinosaurs ruled the earth).  In people it would equal 1/8 of the world’s population (or the population of China in 1981).  In apps it would equal the number of downloads from the iTunes app store in the first nine months of operation.  In dollars, it’s the taxes paid by you and 103,424 of your friends.

A billion packets per second is 3X the performance of Fulcrum’s current generation of FocalPoint switch and it means that Alta-based switches will have the performance needed for applications such as virtualized data center networks.  Virtualization is increasing the density of server farms and allowing data center operators to efficiently deploy cloud services. Virtualization technology is rapidly changing, and is spreading from virtualized servers to virtualized storage, and its being pushed down into hardware by technologies like Intel’s SR-IOV.  The market is attracting acquisition dollars, and also venture capital.  There are a lot of big bets on the future of virtualization.

Those bets can pay off if the network performance is there.  FlexPipe is a pipeline that’s wide enough to support full rate, non-blocking performance for a large number of virtualized flows. And it can deliver these performance levels with all pipeline features enabled. FlexPipe brings with it FocalPoint’s 300ns of latency, which has been on the market for five years and is still five times lower than other data center Ethernet switches.

There’s much more to the FlexPipe, including the ‘Flex’ part, which refers to new programmability features that will be discussed in my next blog.

Comments Welcome: feedback@fulcrummicro.com

By Gary Lee

At Hot Interconnects today, our CTO Uri Cummings outlined the next-generation Alta 40Gb switch chip architecture. Today’s architecture-level discussion gives us the opportunity to focus on key design issues that really make a difference in the performance of networks, but often get lost in the focus on speeds and feeds that occurs when a switch chip is announced.  Fully provisioned throughput, low latency, efficient memory architecture and programmability matter as much (or more) than total port bandwidth.

This is especially true now that there is a sea change in the Ethernet market and the need for high-performance, low latency switching in interconnect applications is expanding as more data centers embrace Ethernet as a compute fabric, and as the economics of Ethernet becomes more essential to building out cloud computing services.

So, what is the Alta architecture?  Alta is a 40Gbps switch with integrated 10GB PHYs that keeps all of the benefits of the FocalPoint switch family, including 200 ns cut-through latency, which is still the lowest latency in an HSE switch to date.  The architecture has full support of data center bridging features, advanced header processing functionality and the parallel-multicast capability that makes it popular in video and Wall Street applications.

Alta marks the debut of a significant performance innovation called FlexPipe™. FlexPipe is the industry’s fastest packet processing engine with performance greater than a billion packets per second even with all pipeline features enabled. But FlexPipe is also programmable, providing the ability to reconfigure various locations in the frame-processing pipeline using microcode.  This enables flexible support for evolving data center standards such as TRILL, FCoE, MPLS and others.  Another feature of FlexPipe is deterministic performance.  FlexPipe is such an innovation that we will break down its features in two upcoming blog posts.

With Alta, FlexPipe is combined with the output queued, shared memory architecture we’ve used in FocalPoint for several years now.  This shared memory design is based on our proprietary RapidArray packet memory.  RapidArray is a key reason for the latency and throughput performance of FocalPoint.  More on this memory design is in another blog post located here.

Alta is designed to support a data center evolution that is moving from 10Gb to 40Gb, moving to support thousands of virtual servers and merging separate data and storage networks into one unified fabric.  Look here for more on Alta later this week and next week, and for the first Alta-based chip announcements in Nov.

Comments Welcome: feedback@fulcrummicro.com

By Fulcrum Microsystems

The Hot Interconnects show is coming up next week, and Fulcrum co-founder and CTO Uri Cummings will use the opportunity to debut the next-generation FocalPoint chip architecture.  More to come about this in this blog, but if you are going to Hot Interconnects, plan to go to Uri’s presentation (Aug. 18 @ 3:15 pm).  Here’s the abstract from the show website:

Introducing the Next Gen FocalPoint Architecture

In 2006, Fulcrum Microsystems launched FocalPoint and set a new standard for Ethernet by delivering the first high-scale, high-density, low-latency 10G switch architecture. FocalPoint ushered in the era of data center Ethernet and opened up the possibility of serving HPC, clustered computing and server interconnect with the same fabric. Since then, the market has taken off and the original architecture has evolved to offer higher scalability, routing, and new data center specific features such as those defined in the IEEE 802.1 DCB Task Group. Ethernet has become firmly established as the leading data-center interconnect technology. But the data center is the place to push the envelope for performance, and Ethernet must evolve both in its throughput and also in its scalability. Uri’s presentation will cover the next-generation FocalPoint architecture, called Alta, which delivers industry’s highest density, line-rate high-speed Ethernet (HSE) switch that comes with a full-speed deterministic programmable pipeline. Come see the future of next-generation data center Ethernet.

By Gary Lee

The market for cloud computing has taken an interesting turn with Amazon’s announcement last month of its Cluster Compute Instances service.  The announcement makes Amazon one of the most high-profile firms to deliver cloud-based HPC service and marks the growth phase of the market for cloud services that depend on high-performance, low-latency networking to deliver their value add.

As reported by Network World, Amazon said its new service provides the same performance as a custom-built infrastructure with pay-as-you-go pricing.  The company said its cluster service delivers up to 10 times the network throughput of its existing EC2 cloud computing services.

As it turns out, many large companies were using the EC2 service for computing jobs that they would normally run on an HPC infrastructure. To respond to this trend, Amazon didn’t change out the computing infrastructure, but rather boosted the performance of its cluster networking gear.

We think many more cloud computing vendors will also make the move to a low latency infrastructure to both keep pace with Amazon and others, but also to help improve their profitability.   If a move to a new infrastructure means a 10X increase in throughput – as Amazon has announced – then that means also that the company can schedule more customer jobs on the same server infrastructure.

Cloud computing performance used to be measured in terms of bandwidth in and out of the datacenter.  That won’t change for some folks, like Facebook, but those companies that sell actual computing cycles will need to consider the impact that a low-latency infrastructure will have on their services.

Comments are welcome: feedback@fulcrummicro.com.

By: Gary Lee

Normally, we limit the focus of this blog to lossless fabrics for telecom and data centers.  But those worlds are ultimately impacted by bandwidth consumed by users like you and me.  So I thought this recent article in Wired was worthy of comment. It discusses how the bandwidth offered by Verizon’s FiOS fiber data services for consumers has been steadily climbing and is now up to 10Gbps to households in some communities.

With HD televisions and triple-play network services requiring more bandwidth per household, service providers are looking at ways to deliver that bandwidth – all the way to today’s current maximum standard of 10-Gbps. Almost all of these networks are becoming packet-based, with Ethernet the dominate layer 2 protocol.

This skyrocketing demand for Ethernet-based data services is not going away, so how long will it be before carriers are looking to even higher data rates?  Look for 40G and then 100G coming to a set top box near you!

Comments are welcome: feedback@fulcrummicro.com.

By Gary Lee

Here in Southern California, we’re used to thrill rides – from Magic Mountain to Disneyland (and sometimes the freeways in between) – there’s something for everyone.  Now, data center managers in need of more bandwidth are about to get their own “TRILL” ride.

TRILL stands for Transparent Interconnection of Lots of Links, which is a new IETF protocol (RFC 5556) just about to emerge from its working group.  Even before the standard was ratified, Cisco last week announced that it would debut its FabricPath protocol, which is described as a pre-standard “superset” of TRILL.

TRILL uses the redundant links to expand network bandwidth.  Therein lies some controversy as it disables Spanning Tree functionality, implying it doesn’t have its own robust loop prevention capabilities.  Some have said that this substitution will lead to “faulty engineering.”

In our opinion, that’s not a significant issue as TRILL will be used in a controlled data center environment and provides such an increase in bandwidth and in data flow in return that its bound to improve throughput.

Cisco’s announcement also raised concerns in a Network World article that the company’s defacto standard will usurp the final IETF version.  However, the data center is still one of the areas where there’s true switching competition and any vendor contending for business there will need to support a truly interoperable standard.

We think that TRILL will be a great enabler of Data Center Bridged networks and are looking forward to participating in the first real TRILL interoperability plug fest.  Buckle up!

Comments are welcome: feedback@fulcrummicro.com.

By Gary Lee

There has been a lot of buzz lately around the potential of the OpenFlow Switching Protocol to be the control plane solution for the data center interconnect.  For those not familiar with OpenFlow, here’s how I described it in a previous post:

OpenFlow is described on its Web site as an open standard that allows researchers to run experimental protocols on existing networks.  The effort began at Stanford in 2008 and has installations at a variety of college campuses across the country.

The latest excitement is mainly due to the software’s potential for reducing both hardware and software complexity, as well as the potential power savings that comes with central control of resource allocation. Managers of large data centers want hardware vendor flexibility along with a simplified — yet extensible — software stack. OpenFlow has the potential to bring them both.

At the same time, the IEEE is defining DCBx as a way for networking equipment to automatically exchange capabilities in order to create a unified data center network configuration, based on common-denominator functionality.

These overlapping approaches provoke the question of which is a better way to configure a large data center fabric.

DCBx appears to be leading the race with several software implementations now available. In addition, OpenFlow controllers will need to work with legacy data center fabric technology in the near term, including DCBx enabled fabric domains, and are therefore focusing on provisioning resources at the network edge. But in the long-term, OpenFlow could provide a central means of controlling and provisioning the entire data center fabric without the need for DCBx. Should be interesting to see how this plays out.

Comments are welcome: feedback@fulcrummicro.com.

By: Gary Lee

There’s a lot of buzz about 100G Ethernet, with customers saying they need it to solve pressing bandwidth shortages in the carrier network and in the data center.  Some vendors are also caught up in the frenzy and are claiming they can deliver it now.

But a recent press release from the IEEE should serve to remind the market that there’s still work to do to make 100G viable.  In fact, the headline from FierceTelecom sums it up nicely: IEEE’s 802.3ba standard sets stage for 40, 100 Gbps Ethernet – with an emphasis on the stage being set.

We predict that in the enterprise and data center, 40G will emerge in a significant way before 100G due to the complexity and cost of designing the switch and NIC chips.  These chips need to aggregate 10 lanes of 10GbE to deliver 100G performance; that’s a complex design that requires costly cable assemblies.  It looks like 4-lanes will be the configuration of choice for the near future as is used today with 40G.  What’s needed is 25G SerDes that are cost effective, but this will require 28nm processes, with production products still years away.

Comments are welcome: feedback@fulcrummicro.com.

By: Gary Lee

In another sign that Ethernet is taking over in many parts of the telecom network, here’s an article from Light Reading.com about how T1s needed for backhaul networks may die as soon as next year, being replaced by Ethernet:

“Wireless network operators are requesting fewer T1s and more Carrier Ethernet connections for mobile backhaul, and could stop ordering T1s entirely as soon as next year, Tower Cloud Inc. CEO Ron Mudry told attendees at the Backhaul Strategies and Core Convergence for Mobile Operators event here today.”

The article goes on to detail the various costs associated with building carrier networks, including fiber trenching, electronics, rent for cell sites, etc.  With the sum of these costs, though, carriers need to be judicious with the types of networks they deploy.

While they can’t always future-proof their networks, carriers must anticipate increases in bandwidth demands from advanced mobile applications and look to implement scalable network solutions that offer clear upgrade paths to 10G, 40G and, eventually, 100G data rates. They must also do this while minimizing costs.

With new and advanced congestion management, QoS and network clock synchronization capabilities, Ethernet will be the packet backhaul technology of the future.

Comments are welcome: feedback@fulcrummicro.com.