After commit things compile again :).
The HTTP bandwidth throttling is not yet implemented. I'll put a
temporary fix back the next commit that just does it the "old way"...
This includes also non-texture requests (not sure if it's worth it to
explicitely separate this data for just textures).
The texture console now prints: HTTP:c/q/a/r
Where, c = number of 'add' request commands in the command queue
(minus the number of 'remove' request in the command queue and not
taking into account the command_being_processed, nor entirely being
thread-safe with regard to adding requests to 'q' or 'a' next: it is
possible that a request is no longer counted in 'c' but not yet is added
to 'q' or 'a'.
q = number of queued (throttled) requests (by the curl thread).
a = number of actually added requests (to the multi handle).
r = last returned value of 'running handles' by libcurl.
Obviously, c and q should be small (0 or 1) most of the time and a and r
should be equal and maxed out. This turns out to be the case.
Replaces LLTextureFetch::getNumHTTPRequests.
Returns AICurlInterface::Stats::running_handles.
This is work in progress that temporarily doesn't compile because
LLTextureFetch::getNumHTTPRequests is still being used somewhere.
This is now necessary since the curl thread no longer syncs with the
main thread: it is possible that a request finishes after a texture
fetch thread was shot down but before curl was stopped, and curl
calling BufferedCurlEasyRequest::processOutput while objects that the
responder uses were already destructed (most notably
LLTextureFetch itself).
This fixes a bug where unref() was called when a state machine was
aborted before it reached bs_initialized. Debug code was added to detect
errors related to that.
In order to run HTTPGetResponder in any thread, I needed direct access
to LLHTTPClient::request, so I had to move that to the header file,
and therefore had to move ERequestAction from LLURLRequest to
LLHTTPClient to avoid include problems.
With this, textures are fetched with no latency: call to
LLHTTPClient::request runs all the way till the state machine is idle
(AICurlEasyRequestStateMachine_waitAdded). There is small delay till the
curl thread wakes up, which then processes the request and opens the url
etc. When the transaction is finished, it calls
AIStateMachine::advance_state(AICurlEasyRequestStateMachine_removed_after_finished)
which subsequently doesn't return until the state machine is completely
finished (bs_killed). The LLURLRequest isn't deleted yet at that point
because the AITimer of the LLURLRequest runs in the main thread: it is
aborted, but only the next time the main thread state engines run that
is deleted and the timer keeps an LLPointer to it's parent, the
LLURLRequest, so only then the LLURLRequest object is destructed. This
however has nothing to do with the texture-bandwidth loop.
Although it should never happen that a file descriptor is suddenly
closed, it appeared that this happens on linux 64bit when using
FMODex... Not really sure how useful this is, but at least now the
viewer just continues to work, as if -say- the socket was closed
remotely. Before the curl thread would go into a tight loop that it
wouldn't recover from until the watchdog thread terminated the viewer.
Add CURLTR debug channel for libcurl API calls,
and use CURLIO only for libcurl debug output.
Note: need to set gDebugCurlTerse to true for
filtering to take effect, then pass 'debug_on'
to the LLHttpClient methods that require debugging.
Adds a std::map for hostname (or urls) --> PerHostRequestQueue
objects. The latter keeps track of the number of added curl easy
requests and decides if a new request should be throttled or
not, as well as provides the queue to queue throttled requests.
At the moment CurlConcurrentConnectionsPerHost is set to 16,
because things really don't work without LL supporting connection
reuse if we limit it to 2. CurlConcurrentConnectionsPerHost is
also set to non-persistent so that we can easily change it in the future
(once we decide on it's final value it can be set to persistent).
Moved AICurlPrivate::Stats to AICurlInterface::Stats and added several
counters to keep track of the number of existing instances of
respectively AICurlEasyRequest, AICurlEasyRequestStateMachine,
BufferedCurlEasyRequest, ResponderBase and
ThreadSafeBufferedCurlEasyRequest.
Rename check_run_count to check_msg_queue, because the whole 'run count'
approach is flawed anyway (the author of libcurl told me that THE way
to check for finished curl handles is to just call curl_multi_info_read
every time: it's extremely fast. Any test that attempts to avoid that
call is nonsense anyway.
The reason the assertion failed might have been caused by the fact
that we're comparing the current number of easy handles with the
number of running handles of 'a while ago'. It is possible that a
easy handle was removed in the meantime.
In order to check if that hypothesis is right, I moved the assertion
to directly below the call to curl_multi_socket_action where it
should hold. If this new assertion doesn't trigger than the hypothesis
was right and this is fixed.
Every curl transaction is a AICurlEasyRequestStateMachine which has a
AICurlEasyRequest as member, which is a reference counting pointer to
a ThreadSafeBufferedCurlEasyRequest. And now BufferedCurlEasyRequest is
derived from CurlEasyRequest which is derived from CurlEasyHandle, but
neither are used separatedly.
* Moved Responder stuff to LLHTTPClient.
* Renamed LLHTTPClient::Responder to LLHTTPClient::ResponderWithResult.
* Deleted LLHTTPClientAdapter and LLHTTPClientInterface.
* Renamed AICurlInterface::TransferInfo to AITransferInfo and moved it
to llhttpclient.h
* Removed 'CURLcode code' argument from completed_headers.
Note that in the code, and still, has_curl_request was always false.
However, instead of deleting all code paths that are only executed
when has_curl_request would be true, I fixed the code to work as
intended with my current implementation; which also results in
LLCurlRequests to never expire. This way things won't break
unexpectedly when this ever changes.
Since on this branch isValid was only called still (the rest was
removed already) to check if the curl download expired, I took
the liberty to rename isValid to hasNotExpired.
Conflicts:
indra/llmessage/llcurl.cpp
indra/llmessage/llcurl.h
indra/newview/app_settings/settings.xml
indra/newview/llappviewer.cpp
indra/newview/llmeshrepository.cpp
Resolved:
indra/llmessage/llcurl.cpp:
Basically removed (not used anyway)
indra/llmessage/llcurl.h:
Basically removed (just includes aiculr.h now)
indra/newview/app_settings/settings.xml:
CurlUseMultipleThreads was remvoved.
CurlMaximumNumberOfHandles and CurlRequestTimeOut
are still in there, but unused at the moment.
indra/newview/llappviewer.cpp:
CurlMaximumNumberOfHandles and CurlRequestTimeOut
are unused at the moment.
indra/newview/llmeshrepository.cpp:
Lock mSignal always (is unlocked inside wait()).
Use mSignal lock to see if we are waiting; remove mWaiting.
Return false from the MeshFetch functions iff we have to retry
a HTTP fetch. Catch the error exception thrown by getByteRange
instead of using it's return value (always returns true
anyway).
