So I had read ESX4 was better about HA setup than 3.x. I was in the beta, but didn't have the chance to test HA. So after upgrading the production cluster to ESX4, HA was broken again.
Enabling HA gave the error:
"Cannot complete the configuration of HA agent on the host. See the task for details for addional information. Misconfiguration in the host network setup."
After poking around for a while, I realized that one host had a bad Default Gateway (off my 1) setup. After correcting that, HA works like a champ.
I did also fix the fact that one of my /etc/hosts files had my VMOTION service console IP and not the correct primary SC IP.
But its all good new..
ESX 4 is really nice..