The Route Less Traveled: Musings on Layer 3 Switching

According to Wikipedia, the only difference between layer 3 switching and routing is in the internal hardware implementation.  Layer 3 switching takes place in specialized hardware vs. general purpose microprocessors in traditional routers. Furthermore “Layer 3 switches can be placed anywhere in the network because they handle high-performance LAN traffic and can cost-effectively replace routers.”  Thus, the motivation behind layer 3 switching appears be a need for high performance routing in our LAN environments.

Traditionally we built our LAN (and WAN for that matter) infrastructure by dividing our IP address space into subnets, which required inter-subnet routing.  The escalation in LAN bandwidth per user and changing application performance requirements placed additional demand on routers.

But let’s not overlook one very important aspect that had a far greater impact:  Over the years we virtualized our LANs into VLANs.  The original motivation for doing this was to control broadcast domains as our layer 2 switching infrastructure grew.

Dividing our LANs up into VLANs led to new benefits in the areas of security, management, and user control to name a few.  We even dedicated VLANs to applications requiring different handling characteristics such as VoIP.  Some esoteric uses for VLANs included assigning multiple SSIDs to a wireless access point in order to map to VLANs on the wired side.  This allowed us to prioritize incoming wireless traffic, restricting uses to certain resources, or both (corporate vs. guests or faculty vs. students, for instance).  It seems that our imagination became the limit.

Unfortunately we were faced with a rather nasty problem - how to get traffic between these isolated broadcast domains?  After all, packets like ARP broadcasts were stopped cold in their tracks. The answer?  Associate a VLAN with a particular subnet and use a router behind a switch to handle VLAN-to-VLAN traffic.

How efficient!  Not.  Extra packet hops and traffic across LAN segments and significant additional load on the router ensued. This problem became affectionately known as "local routing" and all the expert systems in protocol analyzers detected it (well the good ones anyhow).  Two devices could be plugged in side-by-side in the same layer 2 switch but on different VLANs and subnets.  Instead of containing the traffic inside the switch with high-speed switching between ports, the packet was sent up to a router and back down again, wasting bandwidth and router resources.

So what came next?  We threw more backbone bandwidth and router CPU at it!  I'm only half serious here. But for a while it was the only way to solve the problem and some are still doing it this way today.  Layer 3 switching to the rescue.

By pushing layer 3 switching down to the device per port level and applying techniques similar to content addressable memory found on layer 2 switches, but for IP addresses instead of MAC addresses, we solved our little VLAN problem.  Not only that, but we can now have as many ports as we want dedicated to one VLAN/subnet to serve multiple devices.  No more restrictive one-to-one subnet-to-router port relationships.  Slick.

One thing I really like about layer 3 switches when it comes to network analysis and troubleshooting is that they decrement the TTL in the IP header, just like a conventional router.  Thus allows you can see how many store and forward devices a packet has traversed, even if they were virtual hops within a single layer 3 switch.  Try that in conventional, transparently-switched layer 2 environments.  You have to resort to manually figuring out the switch route or using specialized analysis tools to follow the switch MAC address tables via SNMP. Fluke Networks specializes in this sort of thing, as a matter of fact.

As for layer 3 switching vs. routing in your environment, much of it boils down to how many localized VLANs you have and how much traffic passes between them.  This becomes an interesting exercise in analysis because some applications like VoIP will have a dedicated VLAN up to a certain point (think about scalability) and yet other applications and servers will have their own unique switching and routing requirements.
 
Another interesting question is at what point do you push layer 3 switching down to the workgroup?  I’ll leave that as an exercise for the student.