A little over a year ago I decided to get my lab 10Gbps capable. At the time, the Quanta LB6M seemed like the obvious choice with 24x SFP+ interfaces, several 1Gbps copper interfaces and a price tag of only $300 USD. It worked well in my lab, but as you can imagine, older 10Gbps technology is far from energy efficient. The switch idled at close to 130W, which wasn’t far off from all three of my compute hosts put together. It was also very loud with seven high-RPM 40mm fans. Even when at their lowest setting, these chassis and PSU fans were painfully loud. There is no doubt that this switch is more at home in a datacenter than a home lab. It wasn’t long before I started keeping an eye out for newer offerings based on more efficient and cost-effective technologies. That’s when I came across MikroTik’s latest 3xx series “Cloud Router Switches”.
The MikroTik CRS309-1G-8S+IN
MikroTik is a technology firm based out of Latvia and is well known for their unique networking products. They sell a range of switches, routers, wireless products and even their caseless RouterBOARD systems for those interested in doing custom routers.
I was originally looking to purchase the CRS309’s bigger brother – the CRS317. Feature-wise they are very similar, but the CRS317 doubles up on SFP+ and 1Gbps copper interfaces. With two SFP+ ports on each of my four ESXi hosts, the CRS309’s eight SFP+ ports seemed to be sufficient. The feature that sold me on the CRS309 as opposed to the CRS317, however, was its completely passive/fanless design. The CRS317’s fans only spin up if the unit gets hot, but I liked the simplicity of a completely fanless solution.
Feature-wise, the CRS309 is a pretty impressive switch. Its state of the art Marvell Prestera 98DX8208 switching chip (98DX8216 in the CRS317) is what makes this such an efficient unit. The 98DX8208 is highly integrated and is good for line rate forwarding on all the SFP+ ports. It also includes an integrated dual core 32-bit ARM based processor running 800MHz. You can find more information on the Marvell Prestera here. The flash storage is a pretty spartan 16MB, but this seems to be plenty for the RouterOS and SwOS firmware to coexist on the switch simultaneously. I’ll get more into the CRS309’s software in a future post.
From a L2 perspective, it’s a perfectly capable unit. It can do line-rate L1 and L2 forwarding at about 81,000Mbps aggregate, or 162,000 Mbps full-duplex. This is because the 98DX8208 has an ASIC switching component and all L2 forwarding is done in hardware. The L3 features of the switch, however, are all done in software and must be processed by the integrated ARM CPU cores. Based on MikroTik’s test results, about 2.5Gbps can be expected when traffic has to go through the CPU. Your mileage may vary though depending on the features you use. This may not sound great, but in a home lab environment, you don’t really need high throughput for inter-VLAN routing or other L3 features. I’m most interested in having 10Gbps line rate throughput for host-to-host VSAN traffic, iSCSI, vMotion etc. If you really do need faster routing, you could always spin up a VyOS VM and use your hypervisor’s horsepower for that purpose. Given the $269 USD price of the unit, I think it’s awesome that you get a full suite of L3 features even if throughput is somewhat limited.
You can find more information and the specifications of the CRS309 at Mikrotik’s site.
Unboxing the MikroTik CRS309
The CRS309 comes in a very unassuming cardboard box. The unit is well packaged without being wasteful with protective materials.
The switch doesn’t have an internal power supply, but it’s so efficient that a simple AC/DC adapter does the job just fine. Depending on where you buy the switch you will get a North American or European based adapter. The included unit is a 24V 1200mA part good for about 29W.
I won’t be rack mounting this switch, but it does include rack ears. Because of its small size, one of the ears is extra long to allow it to fit in a standard 19” rack.
The unit itself feels very light-weight. The white paint job is somewhat unique and sets it apart from other network vendors – I personally like the look of it. You’ll notice that the 1Gbps copper port can also accept PoE power eliminating the need for a power adapter. I won’t be using this feature, but it could come in handy in some situations.
Cooling is provided by a large passive heatsink at the rear of the unit. It does protrude out the back of the unit an inch or two, but I’ll take that over fans any day.
Inside the unit, we can see a heatpipe cooler on the Marvell 98DX8208. The two heatpipes extend out to the passive heatsink toward the back of the unit. Interestingly, fan header circuitry does exist on the PCB, but a connector was never soldered on. Two 256MB RAM ICs are soldered on the board as well for a total of 512MB.
Initial Impressions
The CRS309 had no problems at all with my inexpensive Cisco 3M passive DAC cables and Solarflare 10Gbps adapters. They all went link-up immediately and appear to be stable. The power LED is blindingly bright unfortunately – bright enough to light up the entire room at night. You’ll never be unsure if the switch is powered up.
There is no power button on the switch like enterprise switch offerings and it begins booting as soon as DC power is applied. The small buzzer emits a short beep when the bootloader finishes its duties and again with two short beeps once the firmware has finished loading. This is useful if you don’t have console access.
Speaking of console access, I’m glad that a small switch like this still includes the usual DB9 connector. I’ve already locked myself out of the switch a few times while tinkering so it can be very handy. A serial cable is not included, but it takes any standard 8-pin straight-through cable. It you have difficulty connecting, remember that the MikroTik switches use 115200 baud by default, not 9600 like most enterprise switches.
MikroTik advertises the maximum power usage of the CRS309 as 23 watts, which would be awesome even at the maximum. To my pleasant surprise however, the switch is extremely energy efficient. I’ve never seen it creep up above 9W and seems to idle at between 7-8W with all ports in use. I suspect this would go up quite a bit if you were using active DAC cables, or SFP+ to RJ45 adapters that require some current. I couldn’t be happier with the switch’s power draw – this really is perfect for a home lab. As you can expect, with only 9W drawn, the passive heatsink is more than enough to keep the CPU cool. It feels only slightly warm to the touch and never seems to get hot even after hours of operation. We’ll see what the use of L3 features will do to power consumption once I start taxing the CPU a bit more.
I’ll provide a more detailed look at the switch’s performance and software features in a future post, but for now, here’s a quick screenshot of an iperf3 test on a couple of VMs with VMXNET3 adapters:
Not too shabby at all for absolutely no tweaking/tuning and some older 10Gbps cards in these hosts.
Conclusion
So far, I’ve been extremely impressed with the CRS309. For $269 USD, it ticks all the right boxes for a home lab. I can’t wait to start digging in more deeply and exploring what MikroTik’s SwOS and RouterOS is cable of.
vswitchzero 