What version of vCloud Director are you running? vCloud Director 5.1.2 addresses an issue like this.
From the 5.1.2 Release Notes under Resolved Issues:
vCenter Server sometimes disconnects from vCloud Director after upgrading to vCloud Director 5.1
Fixed a problem that could cause vCenter Server to sometimes disconnect from vCloud Director after you upgrade vCloud Director.
