I came across an issue today that took me a while to figure out so I thought I'd share to hopefully help someone out someday.
I'm working on RackSpace cloud servers configuring and compiling packages for a web application. We are also using RackConnect, a service from RackSpace that allows the cloud servers to coexist in a private network with dedicated servers. I started with base CentOS 6.0 image and used yum to install a few packages that I would need. Well, I inadvertently installed
NetworkManager which was the source of my problems.
After a reboot, both network interfaces eth0 and eth1 were coming up with the same IP address. Looking at other 'working' cloud servers I could see that only eth1 was configured with the private IP address and eth0 was disabled.
I confirmed with RackSpace support that the contents of /etc/sysconfig/network-scripts/ifcfg-eth* were correct, but NetworkManager doesn't use these so it was configuring them unexpectedly. It was assigning the private IP address to both interfaces creating an address conflict. I could recreate this by doing
/etc/init.d/NetworkManager restart
which would create the address conflict, and then
/etc/init.d/network restart
which would correctly assign the private IP to eth1 and leave eth0 inactive.
Because this will eventually be an application server, there won't be a need for NetworkManager so my fix was to remove NetworkManager from loading during boot.
/sbin/chkconfig --del NetworkManager
I should mention that I've used NetworkManager on laptops and desktops for years and it works great, but it really shouldn't be installed in a server environment that will probably have static addresses anyway.