Have you tried upping or eliminating the Client::OP_HOST_CONNECTION_LIMIT
option?
Because, for example, if you are retrieving a ton of stuff from github.com
it will only use 8 connections at a time by default.
So if you're retrieving a lot of stuff then your requests can really get backed up waiting.
$client->setOption(Client::OP_HOST_CONNECTION_LIMIT, -1); // no limit, probably not advisable
^ hosts tend to frown on you opening hundreds of simultaneous connections and DoS'ing them :)
But you might try upping it from the default of 8.