Singularity-Viewer/SingularityViewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add a PerHostRequestQueue object.

Browse Source 
Adds a std::map for hostname (or urls) --> PerHostRequestQueue
objects. The latter keeps track of the number of added curl easy
requests and decides if a new request should be throttled or
not, as well as provides the queue to queue throttled requests.

At the moment CurlConcurrentConnectionsPerHost is set to 16,
because things really don't work without LL supporting connection
reuse if we limit it to 2. CurlConcurrentConnectionsPerHost is
also set to non-persistent so that we can easily change it in the future
(once we decide on it's final value it can be set to persistent).
This commit is contained in:
Aleric Inglewood
2012-11-07 15:41:50 +01:00
parent cb52e82a60
commit 68cbf31c8b
11 changed files with 414 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
indra/llmessage/aicurlprivate.h
View File

@@ -34,9 +34,11 @@
#include <sstream>
#include "llatomic.h"
#include "llrefcount.h"
#include "aicurlperhost.h"
class AIHTTPHeaders;
class AIHTTPTimeoutPolicy;
class AICurlEasyRequest;
class AICurlEasyRequestStateMachine;
namespace AICurlPrivate {
@@ -361,6 +363,7 @@ class CurlEasyRequest : public CurlEasyHandle {
AIHTTPTimeoutPolicy const* mTimeoutPolicy;
std::string mLowercaseHostname; // Lowercase hostname (canonicalized) extracted from the url.
PerHostRequestQueuePtr mPerHostPtr; // Pointer to the corresponding PerHostRequestQueue.
LLPointer<curlthread::HTTPTimeout> mTimeout;// Timeout administration object associated with last created CurlSocketInfo.
bool mTimeoutIsOrphan; // Set to true when mTimeout is not (yet) associated with a CurlSocketInfo.
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
@@ -402,6 +405,11 @@ class CurlEasyRequest : public CurlEasyHandle {
inline ThreadSafeBufferedCurlEasyRequest* get_lockobj(void);
inline ThreadSafeBufferedCurlEasyRequest const* get_lockobj(void) const;
// PerHost API.
PerHostRequestQueuePtr getPerHostPtr(void); // (Optionally create and) return a pointer to the unique
// PerHostRequestQueue corresponding to mLowercaseHostname.
bool removeFromPerHostQueue(AICurlEasyRequest const&) const; // Remove this request from the per-host queue, if queued at all.
// Returns true if it was queued.
protected:
// Pass events to parent.
/*virtual*/ void added_to_multi_handle(AICurlEasyRequest_wat& curl_easy_request_w);