The Critical Lowdown Podcast Episode 24
Web-Scale Networking Enabling Modern Data Centers
The rapid advancement of digital technologies has led to an unprecedented increase in network demands. Network operators face the ongoing challenges of providing scalability and introducing new services while reducing costs. IP Infusion is leading the charge to provide these solutions. IP Infusion deliver open, standards-based solutions that span from the edge of the network, through the transport layer to multi-cloud environments.
On this episode of the Critical Lowdown, our Head of Technical for EMEA, Barry McGinley sat down with IP Infusion’s Vince Schuele, Director of Technical Solutions for Partners in EMEA and India, to talk about their latest innovations, and the expansion of the feature set for the Data Center with the OcNOS 6.3 release.
Subscribe to The Critical Lowdown from EPS Global wherever you get your podcasts:
Vince Schuele
Director of Technical Solutions for Partners (EMEA and India), IP Infusion
Vince Schuele is the Director of Technical Solutions for Partners in EMEA and India at IP Infusion. He began his career in the US military, where he helped build NATO networks in Europe and spent eight years as a Signal Officer. His experience includes working as a systems integrator, a consulting network architect, and managing satellite communication nodes for US Army Europe. Schuele has also worked on large scale MPLS deployments, secure Data Center deployments, and coalition networking. His time in service concluded with advising on and deploying security strategy for the United States European Command. Currently, his main focus is on service provider and enterprise Data Center deployments. Schuele is passionate about open and disaggregated networking space and is excited to be a leader in the field with IP Infusion.
Barry McGinley
Senior Systems Engineer, EPS Global
Barry McGinley, Senior Systems Engineer at EPS Global is based in Dublin, Ireland and provides the first layer of tech support all customers in EMEA.
Barry works on POCs for Data Centre, Service Provider and Enterprise deployments and leads the adoption of new bare metal hardware for carrier and telecom applications. Barry has in-depth expertise of the disaggregated networking landscape covering all of the open-source and commercial software and hardware on the market. He is a sought-after speaker on the topic, and is the author of a popular tech blog on epsglobal.com. He is an avid Celtic supporter and partial to a pint of Guinness.
Barry has a BSc in Networking Applications and Services from TU Dublin.
If you have any questions about or need advice or tech support for your upcoming project, don’t hesitate to get in touch. Or check out our Service Provider Solutions here.
Transcript of Podcast
Barry: Welcome to another edition of the Critical Lowdown from EPS. I'm Barry McGinley, Head of Technical for EPS and EMEA. Today, we're talking to IP Infusion (IPI), a software company in the open networking or disaggregated space. We've been working with IPI for many years, and today we're discussing their latest software release and what it means for open networking, particularly for datacenter networking. I'm joined by Vince Schuele, Director of Technical Solutions for Partners in EMEA and India from IPI.
Vince: Thanks, Barry, I'm excited to chat about what we have going on in the datacenter and our latest release. I started my career in networking in the US military, helping build out NATO networks in Europe. I then worked as a systems integrator, building ISPs as a consultant before joining IP Infusion. I'm passionate about the open and disaggregated networking space and I'm excited to be a leader in the space with IPI.
Barry: Interesting background, Vince. We've been a partner of EdgeCore for over a decade, which led us to work with IPI. Could you explain a bit about IPI's history and where they are now?
Vince: IP Infusion started over 20 years ago, selling source code known as ZebOS, now called OcNOS Control Plane. We also offer OcNOS Flex for ODMs who want to run IP Infusion without integrating the source code. We have our commercial offering, OcNOS, which we've been focusing on for about eight years now. We integrate it onto Broadcom, Merchant, Silicon with ODM partners such as UfiSpace, EdgeCore, Celestica, Dell, and Delta. We load our software onto these white box switches, turning them into routers that operate like others you may be familiar with.
Barry: You've had some significant successes on the telecom side, with organizations like TIP and OCP. Could you share some of these?
Vince: We've been a Gold Badge member of TIP for several years and have participated in the O-RAN Plugfest. GigaOM has recognized us as a leader in disaggregated networking for three years straight. We've had numerous customer wins, like Netomnia in the UK, who use our Service Provider platforms to deliver connectivity to over a million homes. In the US, we have customers like Vive Broadband, who are big users of our OcNOS Service Provider offerings.
Barry: We've also seen success with major telcos in Africa, like Africell, Comsol, and Afribone in Mali. Now, let's talk about the datacenter side of things. Why focus on the Data Center now?
Vince: Following our success in the Service Provider market, we're moving into the Data Center market. We've always had a presence here, with customers like Lynx in London. We see a lot of exciting opportunities and are focusing on our EVPN deployments, with both MPLS and VXLAN underlays.
Barry: You recently released version 6.3. What new features and hardware platforms did you add for Data Centers, and what's coming next?
Vince: In version 6.3, we added support for a new 10G switch for top of rack based on Trident, and a management switch for out-of-band management. We also added support for ZR, ZR+ optics, and continue to support our Cassini offering, an open optical packet transponder. In upcoming versions, we'll be adding more hardware platforms from Celestica. One of the main advantages of disaggregated networking is the freedom of hardware choice, allowing you to use different ODMs with the same software.
Barry: That's great. I'd like to add that we've had numerous successes with major telecommunications companies in Africa, such as Africell, ComSol, and Afribone in Mali. We've achieved success in various areas of the telecoms network, not just one. Today, we're going to discuss the Data Center aspect. Why the sudden focus on the Data Center? Although you've always had it in your sights, there was a distinct emphasis on the telecom side. So, why return to the Data Center and expand your portfolio in this area now? What's the rationale behind this timing?
Vince: If you've been tracking our progress, you'd have noticed our impressive wins. We reported an 800% growth at the start of the year in one of our press releases. We've started transferring these wins from the Service Provider to the Data Center. We've always had a presence in the Data Center sector. One of our original clients was LINX and their IXP in London. They are a Data Center client we're focusing on expanding. We're taking the success we've had in the Service Provider market and transferring it to the Data Center market. We're broadening our offerings, introducing new switches. We're seeing a lot of enthusiasm and intriguing opportunities. We've invested a significant amount of focus into our EVPN deployments, with both MPLS and VXLAN data points.
Barry: Okay, we'll delve into the use cases in a few minutes. On the Data Center side, I know you guys released 6.3 recently and it generated a lot of buzz. Can you tell us what was added in terms of hardware platforms and software features? Also, what's coming up after 6.3 specifically for Data Centers?
Vince: Great question. We've added two new platforms as part of the 6.3 release. We now support a new 10G switch for top of rack based off Trident, which is a leaf switch with 48x 10G copper ports and 600G uplinks. We also added a management switch with 1000BASE-T for out-of-band management and layer 2/3 functionality. It doesn't support VXLAN or EVPN, but it completes your portfolio by providing support for management, Data Center fabric, and switching fabric. Also, with our Service Provider platforms, if you need MPLS or ZR, ZR+ optics support, we've added that in 6.3. We continue to support our Cassini offering, which is a spine switch that can be used for DCI with CFP2s. It's the first open optical packet transponder and has been very successful. It's based off Tomahawk and serves as a DCI from a spine switch.
Barry: Is the 1G switch from Celestica?
Vince: Yes, it's a Celestica switch. We plan to introduce more Celestica products in the 6.4 and 6.5 releases. This includes not only management but also actual GLEEF switches, where we're trying to implement Tomahawk to diversify our hardware platform choices. We haven't discussed this much, but one of the main benefits of disaggregated networking is the freedom to choose hardware. You can source hardware from various ODMs and implement the same software across all of them.
Barry: I agree. During the COVID pandemic, amidst chip shortages, we could switch from Ufi to EdgeCore and back, thanks to your work on the ComRAN chips for both boxes. This choice not only reduces costs but also makes the 1G switch more affordable. Lately, the prices for 1G switches have been quite high, so it's great to hear about another offering in this area.
Vince: Indeed, and this brings us back to the freedom of choice. In a recent webinar, the top reason participants gave for choosing disaggregated systems was this freedom. As we continue to collaborate with more ODM partners and add more platforms to our releases, we're expanding this freedom.
Barry: Exactly, the concept of open networking provides this choice, which in turn drives down prices of switches and routers for consumers. Moving on to use cases, you mentioned EVPN over VXLAN. Besides the usual Layer 2 and Layer 3 class BGP, what is the most common use case for OcNOS in Data Centers?
Vince: We typically see the spine and leaf topology with BGP overlay VXLAN for the data plane. This allows for multi-tenancy across the board with Layer 2 isolation, Layer 2 VPNs, Anycast gateways, and so on. We support both the top of rack Anycast gateway and the centralized gateway methodology. Most of our success comes from IXP deployments. Links already use EVPN, VXLAN, Fabric, and provide Layer 2 segments for their customers. They can access the route servers, go in and out, or have private interconnects with additional Layer 2 segments between them.
Barry: And what about the telco side? They sometimes have different requirements.
Vince: Absolutely, the advantage of having the same OcNOS between DC and Service Provider has been instrumental. We've been able to use DNx based platforms such as Qumran, Qumran 2, Sumov, Jericho 2, which can be integrated into the same fabric. This means that the operations across the board are uniform, whether you're on the management side, working with Trident or Tomahawk on the DC side, or incorporating what we call a Service Provider route in router into the DC. These platforms have deeper buffers and more route scale. We've enabled the ZR/ZR+ port on several platforms, allowing for the use of coherent optics at 400G. Additionally, we've introduced platforms with internet route scale, capable of handling up to two or three million routes. With the external TCAM option, we can expand this to 20 million routes. This provides flexibility to service not only the Data Center fabric side, but also the internet gateway and the DCI. This adaptability is crucial for our upcoming release, which will feature VXLAN MPLS interworking and multi-site capabilities on our borderless switching.
Barry: I see the plan and the progress made over the past five or six years in managing all areas of the network. You mentioned the 400G ZR+ optics and Cassini. Cassini has been around for a while, stemming from the Telecom Infra project's open optical packet transport project. Will the 400G ZR+ in bare metal replace Cassini? What are the benefits of the 400G ZR+ and ZR optics for Data Center connectivity and other areas?
Vince: As Data Centers move closer to the edge and become more metro-centric, we've been able to integrate line systems functionality into our boxes. For instance, if you're building a 400G ring, you can incorporate the ZR+ to get the distance. With higher launch powers, you can start removing some components from the line system and move that transponder functionality into the routing platform. This simplifies the entire deployment on the line side. This routed optical networking or IP over DWDM reduces the number of components and potential issues, while maintaining throughput. I see many applications for this on the metro side, closer to the edge.
Barry: Yes, and obviously deep Data Center interconnect as well to bring it back. It's a really interesting product. We've done some bits with Cassini that could have been done with this, maybe. The Cassini is a great product with a lot of working parts. You obviously have your optic, your transponder, and so on. They usually come from different companies because it's an open networking product, so it can be quite difficult.
Vince: I think when you really look at the difference between the Cassini and the Open ZRs ZR+, Cassini's has the CFP2 modules and that's the TAI interface. Whereas the pluggables for ZR ZR+ are based off CMIS 5. So you're using the same interface, QSFP-DD, across the board for your gray optics as you'll use for your long-haul optics. You don't have to go and get this special platform. You have the port there, you have the standard space port, pop in a pluggable and you're good to go.
Barry: You can see the benefits of it. So, you have touched on it a little bit, all the different areas that OcNOS are working in. It begs the question about maybe automation across all these areas. What automation tools or platforms are available, specifically for the Data Center?
Vince: Absolutely, automation isn't really optional anymore, especially in the Data Center space. People are always asking about Zero Touch Provisioning (ZTP) and how to provision everything. So, later this year, we're launching a new network management system, IP Maestro. This will be our automation platform going forward. We also support plugins and other methodologies. If you're using Ansible, we publish Ansible modules on Ansible Galaxy. You can visit our GitHub page and download these modules for use. We also publish our Yang models and we're big supporters of OpenConfig and NetConf. The choice of which option to use depends on your current platform and your automation engine. Automation is a complex subject since there are many ways to do it, and everyone customizes it based on their business workflow. We've been flexible in our approach, with NetConf support, Ansible modules, OpenConfig, and our own NMS to enable interaction with any platform. This gives you the same choice for automation that you have with open networking.
Barry: That's great. Will this new NMS work over the telecom products as well?
Vince: It'll work across our entire product portfolio. It's based on NetConf support and data model-driven, so we can push out the intent towards the network with the models and have the deployments that you want to see.
Barry: That's great. As you said, it's something everybody's looking for in the Data Center. It's not a would-like-to-have kind of tool anymore. So, 6.3 was released about a month ago, and 6.3.1 is out now. What features came out in 6.3 that were more groundbreaking than before, and what is coming in future releases?
Vince: I'm going to cover more than just Data Centers here because we've done a lot. Most of this was around bringing our feature parity on our Coomeron 2 platforms and enabling those larger platforms to have SRMPLS capabilities. We've improved our EVPN capabilities so we can do layer 3 VPNs with EVPN. This simplifies operations by enabling the operations teams to remove some of those protocols, make the transitions from LDP based VPLS and VPN v4 for L3 VPN, and move all that into EVPN. We've also expanded our offering to cover not only the larger side with up to 4.8 terabits on the Coomeron 2 with the dense 100G 400G, but also added some fanless options on the smaller side. So now we can hit that full range of operations from access aggregation core and peering transit and Data Center, as well as the long haul with the ZR/ZR+ optics.
Barry: That sounds great. So you've got every area covered. Are you planning to move into any other area, like OLTs or transponders, or are you sticking with Data Centers?
Vince: Right now, we're partnering with technology partners to service other areas. We're really focusing on the Ethernet transport, the MPLS capabilities, and being able to move the packets around to provide highly available VPNs to come with whatever OLT you want. We'll aggregate that, push that back to a BNG, go with a vBNG provider like net elastic or somebody else, and then really start to take advantage of the capabilities of other open networking companies. We're focusing on our strengths, our core business which is routing and switching, and being able to provide quality services and features to both the Service Provider Data Center market.
Barry: From my experience, you certainly do. Now, to finish off, where do you see IPI and open networking in general in four or five years?
Vince: I think we're starting to see the tipping point of open networking and its adoption. As we continue to bring on more platforms and as merchant silicon catches up, the disaggregated space is only just beginning. We're providing the same SRMPLS, EVPN, EVPN multi-homing, SRV6 is coming, and all the merchant silicon now supports that in hardware. We're also continuing to make inroads into the largest providers across the globe. In my opinion, it's only a matter of time before this is considered the standard.
Barry: I see it when I look at the graphs. There's this kind of 10% to 12% to 14% of 'other', and I know what the 'other' is - it's mostly this stuff. Until it starts being called by its proper name - open networking or disaggregation - other people will take notice. We've been evangelizing this for a decade now, and the last two or three years have been hugely successful. So, Vince, thank you very much for your time today. I really appreciate it.
Glossary of Terms