ESXi Multicast between VM’s – Nexus 1000v

I had this irritating issue the last few days, I have 5x Redhat 6.2 VM’s deployed from template, customization script works as expected.
The application needs multicast traffic to pass between VM’s, so I added the iptables rule

-A INPUT -m pkttype –pkt-type multicast -j ACCEPT

iptables

Multicast should work now right? It didn’t…… Queue hours of troubleshooting. Disable iptables, trying E1000, VMXNET3, removing / replace nic etc…

Finally I did a clean install of 2x Redhat 6.2 VM’s multicast works!!! So it must be the templates / customisation script.

So I created a new template using the recommended procedure

To prepare a Red Hat Enterprise Linux virtual machine for use as a template
  1.  Connect to the Red Hat Enterprise Linux 6 virtual machine to be used as a template. Flag the system for reconfiguration by running the following command as root:

    # touch /.unconfigured
  2.  Remove ssh host keys. Run:

    # rm -rf /etc/ssh/ssh_host_*
  3.  Remove the MAC address inherited from the original virtual machine. Before doing so, deactivate the eth0 network interface. Run:

    # ifdown eth0

     Use a text editor to modify the /etc/sysconfig/network-scripts/ifcfg-eth0 file. Remove the HWADDR entry, which is in the format of the following example:

    HWADDR=00:AB:CD:12:34:56

    Save and close the file, then bring the network interface back up. Run:

    # ifup eth0
  4.  Using a text editor, remove the entry for the eth0 network interface in the /etc/udev/rules.d/70-persistent-net.rules file, which is in the format of the following example:
    SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:AB:CD:12:34:56", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
  5.  Shut down the virtual machine. Run:

    # poweroff
    The virtual machine has now been sealed, and is ready to be used as a template for Linux virtual machines.

I deployed 2x VM’s and multicast works! The next day im ready to deploy 5 new VM’s but multi cast stops working again.

Finally the last possible cause is the Nexus 1000V, it was ruled out early on as IGMP snooping is enabled by default but it turns out Multicast addressing destined for 224.0.0/24 in some older code revisions of the N1k will not be passed up to the uplink device due to the local link suppression being performed.

The Nexus 1000v uses (or attempts to use) IGMP snooping by default, which this should in theory reduce the overall load of the switch, by inspecting source/destination headers of multicast traffic, and directing the traffic to the right port – most environments I work with do not have an IGMP querier, or multicast routing in use in their environment, making this feature more problematic than helpful.

To remedy this, we can simply disable IGMP snooping on the Nexus 1000v, which will then flood interfaces with the multicast traffic. This can be done by logging into the VSM, entering the configuration terminal, and issuing the “no ip igmp snooping” command.

So the final fix was to create a new vlan on the 1000V and disable IGMP snooping for that VLAN.

Problem solved and multicast now works reliably.

Hopefully this might help you out.

Cheers, Joe

2 thoughts on “ESXi Multicast between VM’s – Nexus 1000v

  1. Joe,
    Another solution is to enable ip pim sparse mode on an upstream layer 3 device on the VLAN that the VM’s reside on (most likely a 5k or 7k nexus switch). The default action for 2k/5k/7k nexus switches for igmp snooping is to block all unknown multicast packets (except 224.0.0.x). Also, disabling snooping may not be the best solution in a large enterprise if there is alot of multicast traffic enabled, because as you already stated by disabling snooping multicasts become broadcasts.

Leave a Reply

Your email address will not be published. Required fields are marked *