Tuesday, September 23, 2008

Scripted ESX build with 3.5 not working

I've been scripting our builds for a while now, but haven't done one since our 3.5 upgrade - I'm just going through a test one now before we roll out some new hosts, and my script process which was working great before doesn't seem to produce a working config anymore.

The script code for routing looks like this - there is some switch config done as well, but since the output is below of esxcfg-vswitch and such, i left it out.

#Service Console
#Add service console IP address
echo Adding service console IP address
/usr/sbin/esxcfg-vswif -a -i 172.31.8.168 -n 255.255.255.0 -p "Service Console" vswif1
sleep 5

#Kernel Network
echo Adding Kernel Network Address
/usr/sbin/esxcfg-vmknic -a -i 172.31.8.169 -n 255.255.255.0 VMotion
sleep 5

#Service Console Routing
#Add esx route to VMkernel
echo Adding default route to Service Console
/usr/sbin/esxcfg-route 172.31.8.1
sleep 3

#Routing
#add networking info to /etc/sysconfig/network
echo adding route
cat > /etc/sysconfig/network << EOF2
NETWORKING=yes
HOSTNAME=esxt01.domain.com
GATEWAY=172.31.8.1
GATEWAYDEV=vswif1
EOF2
sleep 3

Here's the contents of every file and command I could think of thats even remotely related.

/etc/sysconfig/network-scripts/ifcfg-vswif1
DEVICE=vswif1
MACADDR=00:50:56:48:9e:17
PORTGROUP="Service Console"
BOOTPROTO=static
BROADCAST=172.31.8.255
IPADDR=172.31.8.168
NETMASK=255.255.255.0
ONBOOT=yes

/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=esxt01.domain.com
GATEWAY=172.31.8.1
GATEWAYDEV=vswif1

/etc/hosts

127.0.0.1 localhost.localdomain localhost
172.31.0.251 nts200.domain.com nts200
172.31.0.248 nts203.domain.com nts203
172.31.5.148 nts334.domain.com nts334
172.31.8.168 esxt01.domain.com esxt01

esxcfg-vmknic -l
Interface Port Group IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled
vmk0 VMotion 172.31.8.169 255.255.255.0 172.31.8.255 00:50:56:78:d6:51 1500 disabled true

esxcfg-vswif -l
Name Port Group IP Address Netmask Broadcast Enabled DHCP
vswif1 Service Console 172.31.8.168 255.255.255.0 172.31.8.255 true false

esxcfg-vswitch -l
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
Production 64 3 64 1500 vmnic1

PortGroup Name VLAN ID Used Ports Uplinks
vmnts08 8 0 vmnic1

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
Management 64 5 64 1500 vmnic0

PortGroup Name VLAN ID Used Ports Uplinks
VMotion 8 1 vmnic0
Service Console 8 1 vmnic0

Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
TestPrivateNetwork64 1 64 1500

PortGroup Name VLAN ID Used Ports Uplinks
TestPrivateNetwork 0 0

esxcfg-route -l
VMkernel Routes:
Network Netmask Gateway
172.31.8.0 255.255.255.0 Local Subnet
default 0.0.0.0 172.31.8.1

the route command produces the following, with a big delay (10-15 seconds) before it displays the third line with the default route on it.

kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.31.8.0 * 255.255.255.0 U 0 0 0 vswif1
169.254.0.0 * 255.255.0.0 U 0 0 0 vswif1
default 172.31.8.1 0.0.0.0 UG 0 0 0 vswif1

I just get a "Destination Host Unreachable" when I try to ping anything, which tells me it doesn't know how to route to the network. whiskey tango foxtrot.

No comments: