Have you tried bumpng up the idle to see if it makes any difference? I'll guess that it will now idle. Did the idle rpm change when you disconnected the brake booster? Did it happen to go up?