This adds mStatus, mReason and mContent to ResponderBase
and fills those in instead of passing it to member functions.
The added danger here is that now code can accidently try
to access these variables while they didn't already get a
correct value.
Affected members of ResponderBase (that now have less arguments):
decode_llsd_body, decode_raw_body, completedHeaders,
completed -> httpCompleted, result -> httpSuccess,
errorWithContent and error -> httpFailure.
New API:
ResponderBase::setResult
ResponderBase::getStatus()
ResponderBase::getReason()
ResponderBase::getContent()
ResponderBase::getResponseHeaders() (returns AIHTTPReceivedHeaders though, not LLSD)
ResponderBase::dumpResponse()
ResponderWithCompleted::completeResult
ResponderWithResult::failureResult (previously pubErrorWithContent)
ResponderWithResult::successResult (previously pubResult)
Not implemented:
getHTTPMethod() - use getName() instead which returns the class name of the responder.
completedHeaders() is still called as usual, although you can ignore
it (not implement in a derived responder) and call getResponseHeaders()
instead, provided you implement needsHeaders() and have it return true.
However, classes derived from ResponderHeadersOnly do not have
completedHeaders(), so they still must implement completedHeaders(),
and then call getResponseHeaders() or just access mReceivedHeaders
directly, as usual.
LLHTTPClient::getByteRange no longer throws AICurlNoEasyHandle, instead it returns whether or not it threw, to match the true/false convention of LLCurl.
- In addition, the header param has been shifted to earlier in this function to ease diffings.
ResponderBase now has static bool isGoodStatus, introduced into the Responder base class of LLCurl
Blocking http requests (code using responders derived from
BlockingResponder) *do* call getRaw() and getLLSD to get the body of
server error messages. It is therefore perfectly ok if mStatus !=
HTTP_OK.
This fixes at least one case (crash report 8407), which comes
down to not cleanly informing a responder of failure when the
request url is empty (or so badly formed that it isn't a valid
url). As a result, the statemachine would abort() without
informing the responder - which is bad, sort of.
The previous cases where the responder needed to be informed
of a failure, namely "statemachine timed_out()" and "bad_socket()"
when a socket suddenly becomes bad for unknown reason, have been
replaced with the more general 'aborted()' function, which must
be called before the statemachine calls abort(). Clearly this
has been done for all cases of abort() now, so that if the
llerrs fires again in the future then that would have to be
after the statemachine calls finish(), which is still as "impossible"
as it was - hence the llerrs is still there to make sure.
The reason that this seldom happened on SL, and more often on
opensim, even more often on home-brew test grids, seems plausible:
malformed urls happen more in those cases.
I also took the opportunity to improve the robustness of cases
where the curl error code is checked: it makes no sense to check
what curl gives as error code when an internal error occurred.
The inventory bulk fetch is not thread-safe, so the it doesn't start
right away, causing the approvement not to be honored upon return from
post_approved (formerly post_nb).
This patch renames wantsMoreHTTPReqestsFor to approveHTTPRequestFor,
and has it return NULL or a AIPerService::Approvement object.
The latter is now passed to the CurlEasyHandle object instead of just a
boolean mQueueIfTooMuchBandwidthUsage, and then the Approvement is
honored by the state machine right after the request is actually added
to the command queue.
This should avoid a flood of inventory requests in the case
approveHTTPRequestFor is called multiple times before the main thread
adds the requests to the command queue. I don't think that actually ever
happens, but I added debug code (to find some problem) that is so damn
strictly checking everything that I need to be this precise in order to
do that testing.
Most notably getMesh (the only one possibly using any significant
bandwidth), but in general every type of requests that just have to
happen anyway and in the order they are requested: they are just passed
to the curl thread, but now the curl thread will queue them and hold
back if the (general) service they use is loaded too heavily.
When uploading finishes, but is not detected, the timeout should be for
"reply delay", the time that the server takes before it replies, and not
CurlTimeoutLowSpeedTime. This patch adds code that takes this failure
into account (which happened only ONCE for me on Metropolis while flying
around and using trickle (not sure if that is relevant), so it's not
that likely to improvement anything in practise. Note that it is
detected by an assertion when it happens, so that we can safely assume
it normally never happened on SL).
* Generalized PUT / POST configuration by adding
CurlEasyRequest::setPut, which now also supports keep-alive (which
still isn't used).
* Upload content length is now stored in CurlEasyRequest::mContentLength
* CurlEasyRequest::has_stalled() now return false if it was possbile
that the 'upload finished' detect failed AND calls upload_finished()
itself in that case, so it is no longer 'const'.
* If low speed is detect exactly when the last bytes are being attempted
to be sent (unlikely scenario), then the upload gets 4 more seconds
after which is switches to CurlTimeoutReplyDelay.
* Added EDoesAuthentication and EAllowCompressedReply to replace
booleans, for readability and type-safety, as did EKeepAlive. Note
that this change inverts the meaning of the compression related parameter.
* Unrelated: removed an unnecessary #include "llurlrequest.h" from
llxmlrpcresponder.h
This fixes a bug where unref() was called when a state machine was
aborted before it reached bs_initialized. Debug code was added to detect
errors related to that.
In order to run HTTPGetResponder in any thread, I needed direct access
to LLHTTPClient::request, so I had to move that to the header file,
and therefore had to move ERequestAction from LLURLRequest to
LLHTTPClient to avoid include problems.
With this, textures are fetched with no latency: call to
LLHTTPClient::request runs all the way till the state machine is idle
(AICurlEasyRequestStateMachine_waitAdded). There is small delay till the
curl thread wakes up, which then processes the request and opens the url
etc. When the transaction is finished, it calls
AIStateMachine::advance_state(AICurlEasyRequestStateMachine_removed_after_finished)
which subsequently doesn't return until the state machine is completely
finished (bs_killed). The LLURLRequest isn't deleted yet at that point
because the AITimer of the LLURLRequest runs in the main thread: it is
aborted, but only the next time the main thread state engines run that
is deleted and the timer keeps an LLPointer to it's parent, the
LLURLRequest, so only then the LLURLRequest object is destructed. This
however has nothing to do with the texture-bandwidth loop.
Cookies are now collected during redirects, one the last cookie
of a given name is kept. Cookies are then set by looking for
the right cookie name (instead of what viewer 3 does: just
use the last cookie that was set).
This fixes the merchant outbox.
Add CURLTR debug channel for libcurl API calls,
and use CURLIO only for libcurl debug output.
Note: need to set gDebugCurlTerse to true for
filtering to take effect, then pass 'debug_on'
to the LLHttpClient methods that require debugging.
According to the docs
(http://www.cplusplus.com/reference/iostream/istream/readsome/)
readsome would also set state flag eofbit, but apparently I'm
misinterpreting it. Anyway, using read() and then gcount()
to get the number of bytes does work.
ResponderHeadersOnly is a base class for responders that use
HTTPClient::head or HTTPClient::getHeaderOnly. It already
has a needsHeaders() that return true and only allows for
completedHeaders to be overridden.
I removed the CURLOPT_HEADER option for these cases, because
that only causes the headers to be send to the writeCallback
as if they are part of the body, in addition to the headerCallback;
That gave raise to some confusion for the existing code (ie,
unexpected errors when trying to decode the body as LLSD and
duplicated 'low speed' information for the Timeout policy code.
Moved AICurlPrivate::Stats to AICurlInterface::Stats and added several
counters to keep track of the number of existing instances of
respectively AICurlEasyRequest, AICurlEasyRequestStateMachine,
BufferedCurlEasyRequest, ResponderBase and
ThreadSafeBufferedCurlEasyRequest.
* Moved Responder stuff to LLHTTPClient.
* Renamed LLHTTPClient::Responder to LLHTTPClient::ResponderWithResult.
* Deleted LLHTTPClientAdapter and LLHTTPClientInterface.
* Renamed AICurlInterface::TransferInfo to AITransferInfo and moved it
to llhttpclient.h
* Removed 'CURLcode code' argument from completed_headers.
* Removed LLCurlRequest and replaced it's last usage with LLHTTPClient API calls.
* Deleted dead code.
* Renamed all the get4/post4/put4/getByteRange4 etc, back to their
original name without the '4'.
* Moved LegacyPolledResponder::mCode to ResponderBase::mCode.
* Added a parameter to ResponderBase::finished (and pubError*) to set mCode.
* Renamed ResponderBase::decode_body to decode_llsd_body and added ResponderBase::decode_raw_body.
* Use LegacyPolledResponder::finished instead of LegacyPolledResponder::completed_headers
to set remaining cached values.
* Fixed assertion in case of -DCWDEBUG and upload finish detection failure in case of HEAD
method (mDebugIsGetMethod -> mDebugIsHeadOrGetMethod).
* Add XmlTreeInjector : support for LLXmlTree.
* Split BlockingResponder into BlockingLLSDResponder and BlockingRawResponder.
* Final blocking responders are now: BlockingLLSDPostResponder, BlockingLLSDGetResponder
and BlockingRawGetResponder.
* Added LLHTTPClient::blockingGetRaw
* Got rid of hipporestrequest.* -- and fixed hippogridmanager.cpp to use
LLHTTPClient::blockingGetRaw instead, and fixed llviewermessage.cpp to use
AICurlInterface::ResponderWithCompleted and decode_raw_body instead of
HippoRestHandlerRaw and LLHTTPClient::get4 instead of HippoRestRequest::get5.
Renamed AICurlInterface::Responder to AICurlInterface::ResponderBase,
but without the virtual 'event' methods.
Derived from that: Responder and ReponderWithCompleted, where the
first defines result = 0, ErrorWithContent and error, and the latter
completedRaw and completed.
Added HttpClient::IgnoreBody, derived from Responder and implementing
'result' doing nothing; HttpClient::Ignore is now derived from
IgnoreBody and defines the still pure virtual getHTTPTimeoutPolicy.
Added ResponderBase::decode_body, which is now the sole place
where the code makes the decision wether some response data might be
LLSD or not based on the http status result. Before it just tried
to decode everything as LLSD, which seems a bit nonsense.
ResponderWithCompleted::completed no longer does anything, since
classes derived from ResponderWithCompleted are expected to override it,
or never call it by overriding completedRaw.
Entry point is now ResponderBase::finished = 0, instead of
completedRaw, where ResponderWithCompleted implements finished by
called completedRaw, but Responder doesn't: that directly calls
result/errorWithContent/error. Or, for the hack ResponderAdapter,
the entry points are pubResult/pubErrorWithContent.
Those are now the ONLY public methods, so more confusion.
mFinished is now set in all cases.
As a result of all that, it is no longer possible to accidently
pass a responder to ResponderAdapter that would break because it
expects completed() and completedRaw() to be called.
Added LLBufferArray::writeChannelTo.
Fixed bug for BlockingResponder::body (returned reference to temporary).
LLSDMessage::ResponderAdapter now allows a "timeoutpolicy" name
to be passed (not doing so results in the default timings), so
that the timeout policy of the used responder is retained.
Fixed llfasttimerview.cpp to test LLSDSerialize::fromXML() to return
a positive value instead of non-zero, because it may return -1 when the
parsing fails (three places).
Removed LLHTTPClient::Responder as base class from
LLFloaterRegionDebugConsole completely: it isn't a responder!
Several other responder classes were simplified a bit in order to
compile again with the above changes.
Slightly more robust, adds one boolean to all responders, 99%
of which don't need that though, and an extra call redirection,
but well... We might need it this way when I add the possibility
to abort a transfer.
Moved CURLOPT_ENCODING from CurlEasyRequest::setPost_raw, and
CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST from
CurlResponderBuffer::prepRequest, to LLURLRequest::configure,
enabling the debug setting NoVerifySSLCert for the latter
two to work as follows: old behavior if "NoVerifySSLCert"
is not set, and check neither if it is set. However, if
the (new) bool mIsAuth is set the behavior of LLXMLRPCTransaction::Impl::init
is used. This is so in a next commit we can replace
LLXMLRPCTransaction with LLURLRequest: LLXMLRPCTransaction::Impl::init
will be removed. For the same reason, when the new boolean
mNoCompression is set then CURLOPT_ENCODING is set to "identity",
otherwise the old behavior (of clearing it) is used.
Introduces AIHTTPTimeoutPolicy objects which do not just
specify a single "timeout" in seconds, but a plethora of
timings related to the life cycle of the average HTTP
transaction.
This knowledge is that moved to the Responder being
used instead of floating constants hardcoded in the
callers of http requests. This assumes that the same
timeout policy is wanted for each transaction that
uses the same Responder, which can be enforced is needed.
I added a AIHTTPTimeoutPolicy for EVERY responder,
only to make it easier later to tune timeout values
and/or to get feedback about which responder runs
into HTTP errors in debug output (especially time outs),
so that they can be tuned later. If we already understood
exactly what we were doing then most responders could
have been left alone and just return the default timeout
policy: by far most timeout policies are just a copy
of the default policy, currently.
This commit is not finished... It's a work in progress
(viewer runs fine with it though).
CurlEasyHandle::mErrorBuffer (CURLOPT_ERRORBUFFER) can NOT be
used to retrieve information about an error returned by
curl_multi_info_read in CURLMsg::data::result. This buffer is
only initialized when a curl_easy_* call returns an error,
and those errors are already printed automagically.
Initialize the buffer with an empty string upon invokation
of an curl_easy_* call, so we are sure the error belongs to
the last call.
This creates a separate events interface structure
for CurlResponderBuffer (AICurlResponderBufferEvents)
for dealing with received HTTP headers.
The headers are passed to the Responder, but only
if the class derived from Responder implements
completedHeaders (otherwise it makes little sense
to even decode the headers).
Basically this is a reimplementation of the functionality
of the old LLHTTPClientURLAdaptor class.
