[Solved]-Best practice of django + PyMongo pooling?


How does connection pooling work in PyMongo?

Every Connection instance has built-in connection pooling. By default,
each thread gets its own socket reserved on its first operation. Those
sockets are held until end_request() is called by that thread.

Calling end_request() allows the socket to be returned to the pool,
and to be used by other threads instead of creating a new socket.
Judicious use of this method is important for applications with many
threads or with long running threads that make few calls to PyMongo

Alternatively, a Connection created with auto_start_request=False will
share sockets (safely) among all threads.

I think it comes down to the type of application you have and how long the requests will hold onto a connection. The idea of calling end_request helps with long running requests holding on to a socket for a long time and causing many sockets to get created. If a single request can release the connection when it no longer needs it, then the socket can be repurposed for other requests.

If they are fast requests, then I believe the auto_start_request=False works by reusing the socket.

Ensuring a connection keeps using the same socket means that is will have consistent reads. Think if you made a query but it got delayed, and then immeditely made another query and it used a different socket. This socket manages to respond before the previous. You would have inconsistent data since it does not reflect the previous write.


Leave a comment