Monday, 2 February 2009

Using ESX 3.5 vmware-vim-cmd instead of vimsh


For those of you familiar with vimsh and used it to configure a scripted install of ESX 3.5, have you noticed that the following error would occur when launching commands using /usr/bin/vimsh ?

/usr/bin/vimsh -n -e "hostsvc/maintenance_mode_enter

Alternatively, by using the wrapper developed for ESX 3.5, vmware-vim-cmd, you would get the following:

/usr/bin/vmware-vim-cmd hostsvc/maintenance_mode_enter

The two commands are detailed in the Xtravirt whitepapers, vimsh and vimsh for ESX 3.5. I would recommend at least having a quick browse to see what can be achieved with these commands. Using vmware-vim-cmd in conjunction with esxcfg- can achieve some very interesting results, especially if you love to create the perfect KickStart build script.

If only it is possible to launch vmware-vim-cmd commands using the RCLI just as esxcfg- can be launched using vicfg-. Anyone have an idea?

A few more examples

Refreshing the network settings
/usr/bin/vmware-vim-cmd hostsvc/net/refresh

Refreshing the storage
/usr/bin/vmware-vim-cmd hostsvc/storage/refresh

The all important enabling VMotion
/usr/bin/vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0

And how about setting vSwitch1 to use Route Based on IP Hash?
/usr/bin/vmware-vim-cmd hostsvc/net/vswitch_setpolicy --nicteaming-policy=loadbalance_ip vSwitch1

And setting vSwitch0 to use Route Based on the Originating Virtual PortID. (vSwitch0 has two portgroups using VLAN tagging, 1 for Service Console and 1 for VMotion, we wish to use active-passive nic teaming policy)

Set active vmnic0 and standby vmnic2 for Service Console
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicorderpolicy-active=vmnic0 vSwitch0 'Service Console'
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicorderpolicy-standby=vmnic2 vSwitch0 'Service Console'

Set active vmnic2 and standby vmnic0 for VMkernel network
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicorderpolicy-active=vmnic2 vSwitch0 VMkernel
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicorderpolicy-standby=vmnic0 vSwitch0 VMkernel

Set vSwitch overide load balancing policy
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicteaming-policy=loadbalance_srcid vSwitch0 'Service Console'
/usr/bin/vmware-vim-cmd hostsvc/net/portgroup_set --nicteaming-policy=loadbalance_srcid vSwitch0 VMkernel

Let's not forget to refresh our network settings
/usr/bin/vmware-vim-cmd hostsvc/net/refresh
/usr/bin/vmware-vim-cmd internalsvc/refresh_network


Chris said...

Two questions:

1. What's the difference between these two?

vmware-vim-cmd hostsvc/net/refresh
vmware-vim-cmd internalsvc/refresh_network

The xtravirt doc only lists the internalsvc/refresh_network as what's needed in place of a service mgmt-vmware restart. But I see other blogs using both as you have here.

2. I've also seen other blogs regarding setting an active/passive nic teaming policy for the Service Console and VMotion portgroups. Is this something that's recommended when using "Route Based on the Originating Virtual PortID" and also when when using "Route Based on ip Hash"? And if so, what's the thinking behind this - are they just not supported on active/active configs? (I use "Route Based on ip Hash" as I have a cross stack nic teaming configuration on Cisco 3120 switches).


Hugo said...

Q1. From what I gather the hostsvc/net/refresh will display the "Refresh Network Information" message under the Recent Tasks pane on VI Client directly connected to the ESX host. The internalsvc/refresh_network will NOT.

I can only assume that hostsvc/net/refresh will do the job and is recommended because it is exactly the same as using VI Client to click on "Refresh" under Hardware/Networking/.

Both lines are from Duncan's blog at yellowbricks, I've posted a comment there for clarification.

Q2. They are supported on active/active configs, but the reasoning behind forcing the SC and VK portgroups to use a particular network interface is that "Route based on IP hash" policy will use the source IP and the destination IP to select the uplink network interface. Because vSwitch0 will only route traffic for the SC and VK (each being a unique IP address), you will find that most of the time all traffic will traverse the single network interface, leaving the other underutilised.

"Route based on IP hash" is best used for the virtual machine networks, where you will have multiple virtual machines, hence multiple IP addresses.

Both configs require static LACP 802.3ad enabled channels on the Cisco 3120s.

I'll create a post detailing the designs that I use for vSwitch0 with SC & VK VLAN tagging.