Adds throttling based on on average bandwidth usage per HTTP service.
Since only HTTP textures are using this, they are still starved by other
services like inventory and mesh dowloads. Also, it will be needed to
move the maximum number of connections per service the to the PerService
class, and dynamically tune them: reducing the number of connections is
the first thing to do when using too much bandwidth.
I also added a graph for HTTP texture bandwidth to the stats floater.
For some reason the average bandwidth (over 1 second) look almost like
scattered noise... weird for something that is averaged...
Adds a std::map for hostname (or urls) --> PerHostRequestQueue
objects. The latter keeps track of the number of added curl easy
requests and decides if a new request should be throttled or
not, as well as provides the queue to queue throttled requests.
At the moment CurlConcurrentConnectionsPerHost is set to 16,
because things really don't work without LL supporting connection
reuse if we limit it to 2. CurlConcurrentConnectionsPerHost is
also set to non-persistent so that we can easily change it in the future
(once we decide on it's final value it can be set to persistent).
* Moved Responder stuff to LLHTTPClient.
* Renamed LLHTTPClient::Responder to LLHTTPClient::ResponderWithResult.
* Deleted LLHTTPClientAdapter and LLHTTPClientInterface.
* Renamed AICurlInterface::TransferInfo to AITransferInfo and moved it
to llhttpclient.h
* Removed 'CURLcode code' argument from completed_headers.
* Removed LLCurlRequest and replaced it's last usage with LLHTTPClient API calls.
* Deleted dead code.
* Renamed all the get4/post4/put4/getByteRange4 etc, back to their
original name without the '4'.
* Moved LegacyPolledResponder::mCode to ResponderBase::mCode.
* Added a parameter to ResponderBase::finished (and pubError*) to set mCode.
* Renamed ResponderBase::decode_body to decode_llsd_body and added ResponderBase::decode_raw_body.
* Use LegacyPolledResponder::finished instead of LegacyPolledResponder::completed_headers
to set remaining cached values.
* Fixed assertion in case of -DCWDEBUG and upload finish detection failure in case of HEAD
method (mDebugIsGetMethod -> mDebugIsHeadOrGetMethod).
* Add XmlTreeInjector : support for LLXmlTree.
* Split BlockingResponder into BlockingLLSDResponder and BlockingRawResponder.
* Final blocking responders are now: BlockingLLSDPostResponder, BlockingLLSDGetResponder
and BlockingRawGetResponder.
* Added LLHTTPClient::blockingGetRaw
* Got rid of hipporestrequest.* -- and fixed hippogridmanager.cpp to use
LLHTTPClient::blockingGetRaw instead, and fixed llviewermessage.cpp to use
AICurlInterface::ResponderWithCompleted and decode_raw_body instead of
HippoRestHandlerRaw and LLHTTPClient::get4 instead of HippoRestRequest::get5.
Introduces AIHTTPTimeoutPolicy objects which do not just
specify a single "timeout" in seconds, but a plethora of
timings related to the life cycle of the average HTTP
transaction.
This knowledge is that moved to the Responder being
used instead of floating constants hardcoded in the
callers of http requests. This assumes that the same
timeout policy is wanted for each transaction that
uses the same Responder, which can be enforced is needed.
I added a AIHTTPTimeoutPolicy for EVERY responder,
only to make it easier later to tune timeout values
and/or to get feedback about which responder runs
into HTTP errors in debug output (especially time outs),
so that they can be tuned later. If we already understood
exactly what we were doing then most responders could
have been left alone and just return the default timeout
policy: by far most timeout policies are just a copy
of the default policy, currently.
This commit is not finished... It's a work in progress
(viewer runs fine with it though).
renamed cwdebug/debug_libcurl.cc -> llmessage/debug_libcurl.cpp
and cwdebug/debug_libcurl.h -> llmessage/debug_libcurl.h,
because debug_libcurl.cpp does curl calls that do ares and
openssl calls, so we need to link with those libraries.
llmessage is already linking with those libraries, and contains
the main entry point aicurl.h, so it's a suitable place to put
this.
Bug fix: must always include llpreprocessor.h before including
curl/curl.h.
Bug fix: Added #include "debug_libcurl.h" to hipporestrequest.cpp
and llurlsimstring.cpp which I missed before because they
included "curl/curl.h" instead of <curl/curl.h>. Same in
llwaterparammanager.cpp, but removed include there because it
isn't needed.
Now test DEBUG_CURLIO before including debug_curlio, that
seems better, because otherwise it would make more sense to
replace all #include <curl/curl.h> with #include "mycurl.h"
and then do it there-- but I didn't want to do that.
Bug fix: we undef-ed CURLOPT_DNS_USE_GLOBAL_CACHE, while really
that is an enum, not a macro.
Fixed DEBUG_WINDOWS_CODE_ON_LINUX again by adding a hack for
ioctlsocket(), not instantiating dumb_socketpair unless
DEBUG_WINDOWS_CODE_ON_LINUX is defined and removing again ^M's
introduced with the new windows non-blocking code.
Also changed the type of flags passed to fcntl to int (was long).
This code has been in the viewer source for a long time,
and hasn't been used for a long time (furtherest back that
I checked was Snowglobe 1.4).
Most notably, this removes LLContextURLExtractor and code
that used it because that required an API where AICurlEasyHandle
is created before an url is known, which gets in the way of
reusing connections.
Conflicts:
indra/llmessage/llcurl.cpp
indra/llmessage/llcurl.h
indra/newview/app_settings/settings.xml
indra/newview/llappviewer.cpp
indra/newview/llmeshrepository.cpp
Resolved:
indra/llmessage/llcurl.cpp:
Basically removed (not used anyway)
indra/llmessage/llcurl.h:
Basically removed (just includes aiculr.h now)
indra/newview/app_settings/settings.xml:
CurlUseMultipleThreads was remvoved.
CurlMaximumNumberOfHandles and CurlRequestTimeOut
are still in there, but unused at the moment.
indra/newview/llappviewer.cpp:
CurlMaximumNumberOfHandles and CurlRequestTimeOut
are unused at the moment.
indra/newview/llmeshrepository.cpp:
Lock mSignal always (is unlocked inside wait()).
Use mSignal lock to see if we are waiting; remove mWaiting.
Return false from the MeshFetch functions iff we have to retry
a HTTP fetch. Catch the error exception thrown by getByteRange
instead of using it's return value (always returns true
anyway).
Most in particular llevents.h, which comes along with
the demand that the old events in llevent.h are put
in a namespace LLOldEvents. Made all changes necessary
to compile the rest of the code again (without changing
the actual code: it's still using the old events).
This patch also removes LLStopWhenHandled and LLStandardSignal
from indra/llui/llnotifications.h because those are
moved to llevents.h. That seems to be the only change
to indra/llui/llnotifications.h that isn't floater related,
so I left the rest of that file alone.
Excluded llareslistener, as that appears to only be present for unit-testing
Excluded new SSL methods because, well, they don't work right reliably in v2 for me
That's quite a piece of work. I have NO idea how Jess can downplay the
effort that went into this.
And it's still not complete, local chat is not affected, perhaps group
chat too.
