VMware View 4.6 no longer provisions, refreshes or sends any commands to vCenter 4.1 server even though all status’ are reported as being up

I ran into an issue last night after receiving a call from the same client who had an issue with their View 4.6 environment a few days ago that desktops are no longer being refreshed and all of desktops who have had users logged off of were now stuck in maintenance mode.  The first thought I had was that vCenter probably went down and therefore the commands View was setting over to vCenter were no longer being executed.  Logging into the environment and opening the View Administrator Console webpage showed that there was indeed a red down arrow for vCenter.  Hopping over to vCenter showed that the vpxd service was down and a quick restart got it back up.  Further reviewing the logs on the SQL cluster and the vCenter server showed that the cluster apparently crashed at around 3:49p.m. but recovered by failing over to the other node but vCenter eventually stopped at around 4:30p.m.  No problem, the services are all back up now and all arrows are now pointing upwards and green so I expected to see View actions popping up in the recent tasks window in vCenter.  10 minutes have passed and nothing appears to be happening so I go ahead and force refresh, still nothing. I proceed an go create a test pool, View creates the folder but then stops.  Reviewing the event logs, View Composer logs, View Connection logs showed no errors.  It’s as if either the commands aren’t being sent or there is something wrong with the composer service.  After trying several items such as reinstalling composer, re-entering service account credentials without any luck, I opened a call to VMware.

Fast forward to an hour later when I finally got an engineer on the line because of how busy they were, the engineer spent about an hour clicking around and checking everything I’ve reviewed and said nothing looks wrong.  Seeing how we’re going no where, he reached out to his senior engineer then came back after 30 minutes and told me to shut both of the View connection servers down (the environment had 2).  He then proceeded to wait 5 minutes, power up the first server, wait another 5 minutes after it was started, then boot the second one up.  Within 10 minutes of both servers being up, the desktops started getting refreshed.  His explanation was that this appears to happen to View 4 and 5 environments with more than 1 connection server.  He’s not sure why but it may be the ADAM database needing time to start up, queue commands then send it off which was why we waited.

Not exactly what I expected but I’m glad this worked and I hope this post with no screenshots may be able to help anyone who finds themselves in the same situation.