The behavior of Memcache::connect() is to always reinitialize the pool from scratch regardless of any previous calls to addServer().
E.g.
$mmc = new Memcache()
$mmc->addServer('node1', 11211);
$mmc->addServer('node2', 11211);
$mmc->addServer('node3', 11211);
$mmc->connect('node1', 11211);
The last connect() call clears out the pool and then add and connect node1:11211 making it the only server.
If you want a pool of memcache servers, do not use the connect() function.
If you only want a single memcache server then there is no need to use the addServer() function.