Comparing the Broadcom Silicons used in Datacenter switches

Broadcom is the market leader for switching merchant silicons used in network switches. There are other merchant silicon vendors such as Mellanox, Marvel, EZChip, microsemi and Cavium, but Broadcom is very widely used and adopted in network switches by hardware manufacturers. 

From the merchant silicons, only Broadcom and Mellanox have OCP Certified products. Edge-Core has a Cavium based switch which is under OCP review.

On 10G , 40G and 100G Broadcom has 2 main line of products:

1- Strata XGS Family includes the following silicons (we omit some other models in its family which are not widely used as of 2016 June)

• Trident 2 (10g / 40G)
• Trident 2+ (10G / 40G , Enhanced Trident 2 , with VXLAN Routing)
• Tomahawk (100G)

This family is used in many switches, almost 90% of leaf and spine switches which are based on merchant silicon are using this ASIC family. This family has robust and cost effective ASICs.

The buffer size is between 12MB to 16MB (Tomahawk) , and tables are between 64k to 72k for IPv4 and 20k to 36k for IPv6.

2- Strata DNX Family (Also known as Dune family)

• Qumran (10G/40G)
• Jericho (packet processor)
• FE 3600 (Fabric)

This family is pretty new and is based on Broadcom acquisition of Dune networks. Feature wise, DNX comes from a different architecture has some benefits and advantages :

1. Expandable TCAM : Broadcom Qumran supports external TCAM chip which can be added as an external chip connected to Qumran. The external TCAM chip such as BCM52311 provides a very deep table capacity to Qumran. 
2. Expandable Packet Buffer : Broadcom Qumran utilizes an external DDR4 memory as buffer, allowing a buffer size above 8GB.

Why More Buffer ?

Comparing the StrataDNX's Qumran and StrataXGS's Trident 2 , the Qumran provides Gigabytes of buffer comparing to 12MB buffer of Trident 2.

The more buffer comes to play only when there are devices connetced at different speed to the switch. For example you have a IP Storage box connected via 40Gbps to the switch, and a server connected via 10Gbps. Assuming a high performance cache on storage box, if storage is able to pump 40Gbps of traffic to the switch, the switch will need to use buffer to provide a line rate speed to both server @10Gbps and Storage @40Gbps.

You need to see if you have such kind of requirement in your network where you need a such buffer to handle the speed mismatch between devices. 

Why Large TCAM Tables?

Switch ASICs use tables to do its lookup to find where they need to send each packet. The tables have different names and purposes such as TCAM, FIB, FDB, NetFilter, etc. The table sizes in Trident 2, is between 64k to 72k for IPv4 and 20k to 36k for IPv6. In a standard datacenter or service provider normally this table size is enough to have less ASK-CPU cycles between ASIC and CPU. however, with large table size you can load the full internet routing table inside the switch hardware. Please note that with or without large tables the routing process is still running on CPU (not the ASIC), and it keeps synchronizing the CPU routing table with hardware tables. 

In my opinion, large table size may be beneficial if the switch is being used for ISP connectivity, WAN and have multiple ISP connection.

Well, Broadcom's publish very little information about the chip details. it's not easy to findout what's going on and what are the available products. I have collected all bits and pieces of information from different places and aggregated them in below table:

  Who uses what ?

in our last post where we covered available switches from different vendors, almost all the switches were based on Trident 2 or Tomahawk ASICs.

StrataDNX is used in Arista 's new Universal Leaf and Spine switches , as well as Edge-Core AS5900 switch. If you are interested more on hardware deisgn of the Qumran switch , have a look at this interesting PDF file.