According to the docs
(http://www.cplusplus.com/reference/iostream/istream/readsome/)
readsome would also set state flag eofbit, but apparently I'm
misinterpreting it. Anyway, using read() and then gcount()
to get the number of bytes does work.
ResponderHeadersOnly is a base class for responders that use
HTTPClient::head or HTTPClient::getHeaderOnly. It already
has a needsHeaders() that return true and only allows for
completedHeaders to be overridden.
I removed the CURLOPT_HEADER option for these cases, because
that only causes the headers to be send to the writeCallback
as if they are part of the body, in addition to the headerCallback;
That gave raise to some confusion for the existing code (ie,
unexpected errors when trying to decode the body as LLSD and
duplicated 'low speed' information for the Timeout policy code.
The AIBufferedCurlEasyRequestEvents are not triggered unless
the derived class return true for needsHeaders().
That means, every class that implements received_HTTP_header(),
received_header() or completed_headers(), or implement
the virtual function completedHeaders(), or use the protected
member mReceivedHeaders directly.
This commits adds missing needsHeaders() for LLAvatarNameCache
(thanks Siana) and XMLRPCResponder. The former now uses
mReceivedHeaders directly instead of making a copy.
Adds a std::map for hostname (or urls) --> PerHostRequestQueue
objects. The latter keeps track of the number of added curl easy
requests and decides if a new request should be throttled or
not, as well as provides the queue to queue throttled requests.
At the moment CurlConcurrentConnectionsPerHost is set to 16,
because things really don't work without LL supporting connection
reuse if we limit it to 2. CurlConcurrentConnectionsPerHost is
also set to non-persistent so that we can easily change it in the future
(once we decide on it's final value it can be set to persistent).
Moved AICurlPrivate::Stats to AICurlInterface::Stats and added several
counters to keep track of the number of existing instances of
respectively AICurlEasyRequest, AICurlEasyRequestStateMachine,
BufferedCurlEasyRequest, ResponderBase and
ThreadSafeBufferedCurlEasyRequest.
Since we changed CurlResponderBuffer to be derived from CurlEasyRequest
and therefore changed it's name to BufferedCurlEasyRequest, we should
also rename AICurlResponderBufferEvents to
AIBufferedCurlEasyRequestEvents.
This commit also fixes C++ comment in several places to reflex the
previous name change.
Rename check_run_count to check_msg_queue, because the whole 'run count'
approach is flawed anyway (the author of libcurl told me that THE way
to check for finished curl handles is to just call curl_multi_info_read
every time: it's extremely fast. Any test that attempts to avoid that
call is nonsense anyway.
The reason the assertion failed might have been caused by the fact
that we're comparing the current number of easy handles with the
number of running handles of 'a while ago'. It is possible that a
easy handle was removed in the meantime.
In order to check if that hypothesis is right, I moved the assertion
to directly below the call to curl_multi_socket_action where it
should hold. If this new assertion doesn't trigger than the hypothesis
was right and this is fixed.
Every curl transaction is a AICurlEasyRequestStateMachine which has a
AICurlEasyRequest as member, which is a reference counting pointer to
a ThreadSafeBufferedCurlEasyRequest. And now BufferedCurlEasyRequest is
derived from CurlEasyRequest which is derived from CurlEasyHandle, but
neither are used separatedly.
* Moved Responder stuff to LLHTTPClient.
* Renamed LLHTTPClient::Responder to LLHTTPClient::ResponderWithResult.
* Deleted LLHTTPClientAdapter and LLHTTPClientInterface.
* Renamed AICurlInterface::TransferInfo to AITransferInfo and moved it
to llhttpclient.h
* Removed 'CURLcode code' argument from completed_headers.
* Removed LLCurlRequest and replaced it's last usage with LLHTTPClient API calls.
* Deleted dead code.
* Renamed all the get4/post4/put4/getByteRange4 etc, back to their
original name without the '4'.
* Moved LegacyPolledResponder::mCode to ResponderBase::mCode.
* Added a parameter to ResponderBase::finished (and pubError*) to set mCode.
* Renamed ResponderBase::decode_body to decode_llsd_body and added ResponderBase::decode_raw_body.
* Use LegacyPolledResponder::finished instead of LegacyPolledResponder::completed_headers
to set remaining cached values.
* Fixed assertion in case of -DCWDEBUG and upload finish detection failure in case of HEAD
method (mDebugIsGetMethod -> mDebugIsHeadOrGetMethod).
* Add XmlTreeInjector : support for LLXmlTree.
* Split BlockingResponder into BlockingLLSDResponder and BlockingRawResponder.
* Final blocking responders are now: BlockingLLSDPostResponder, BlockingLLSDGetResponder
and BlockingRawGetResponder.
* Added LLHTTPClient::blockingGetRaw
* Got rid of hipporestrequest.* -- and fixed hippogridmanager.cpp to use
LLHTTPClient::blockingGetRaw instead, and fixed llviewermessage.cpp to use
AICurlInterface::ResponderWithCompleted and decode_raw_body instead of
HippoRestHandlerRaw and LLHTTPClient::get4 instead of HippoRestRequest::get5.
Renamed AICurlInterface::Responder to AICurlInterface::ResponderBase,
but without the virtual 'event' methods.
Derived from that: Responder and ReponderWithCompleted, where the
first defines result = 0, ErrorWithContent and error, and the latter
completedRaw and completed.
Added HttpClient::IgnoreBody, derived from Responder and implementing
'result' doing nothing; HttpClient::Ignore is now derived from
IgnoreBody and defines the still pure virtual getHTTPTimeoutPolicy.
Added ResponderBase::decode_body, which is now the sole place
where the code makes the decision wether some response data might be
LLSD or not based on the http status result. Before it just tried
to decode everything as LLSD, which seems a bit nonsense.
ResponderWithCompleted::completed no longer does anything, since
classes derived from ResponderWithCompleted are expected to override it,
or never call it by overriding completedRaw.
Entry point is now ResponderBase::finished = 0, instead of
completedRaw, where ResponderWithCompleted implements finished by
called completedRaw, but Responder doesn't: that directly calls
result/errorWithContent/error. Or, for the hack ResponderAdapter,
the entry points are pubResult/pubErrorWithContent.
Those are now the ONLY public methods, so more confusion.
mFinished is now set in all cases.
As a result of all that, it is no longer possible to accidently
pass a responder to ResponderAdapter that would break because it
expects completed() and completedRaw() to be called.
Added LLBufferArray::writeChannelTo.
Fixed bug for BlockingResponder::body (returned reference to temporary).
LLSDMessage::ResponderAdapter now allows a "timeoutpolicy" name
to be passed (not doing so results in the default timings), so
that the timeout policy of the used responder is retained.
Fixed llfasttimerview.cpp to test LLSDSerialize::fromXML() to return
a positive value instead of non-zero, because it may return -1 when the
parsing fails (three places).
Removed LLHTTPClient::Responder as base class from
LLFloaterRegionDebugConsole completely: it isn't a responder!
Several other responder classes were simplified a bit in order to
compile again with the above changes.
Slightly more robust, adds one boolean to all responders, 99%
of which don't need that though, and an extra call redirection,
but well... We might need it this way when I add the possibility
to abort a transfer.
This fixes the problem that existed with received headers:
The server sends some headers ("set-cookie") more than once
in the same reply, which cannot be stored in std::map.
The old code just ignored the additional cookies, while
curlthreading3 (since the introduction of AIHTTPHeaders)
caused an assertion.
AIHTTPReceivedHeaders is written around a std::multimap
and allows to retrieve multiple headers with the same key.
Also, it is case insensitive so that if a server sends
"Content-Type" it will still find it (the viewer looks for
"content-type").
Minor changes like comment fixes and addition of accessors
that will be needed for future commits.
Also removed Responder::fatalError as it was never used.
Moved CURLOPT_ENCODING from CurlEasyRequest::setPost_raw, and
CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST from
CurlResponderBuffer::prepRequest, to LLURLRequest::configure,
enabling the debug setting NoVerifySSLCert for the latter
two to work as follows: old behavior if "NoVerifySSLCert"
is not set, and check neither if it is set. However, if
the (new) bool mIsAuth is set the behavior of LLXMLRPCTransaction::Impl::init
is used. This is so in a next commit we can replace
LLXMLRPCTransaction with LLURLRequest: LLXMLRPCTransaction::Impl::init
will be removed. For the same reason, when the new boolean
mNoCompression is set then CURLOPT_ENCODING is set to "identity",
otherwise the old behavior (of clearing it) is used.
Introduces AIHTTPTimeoutPolicy objects which do not just
specify a single "timeout" in seconds, but a plethora of
timings related to the life cycle of the average HTTP
transaction.
This knowledge is that moved to the Responder being
used instead of floating constants hardcoded in the
callers of http requests. This assumes that the same
timeout policy is wanted for each transaction that
uses the same Responder, which can be enforced is needed.
I added a AIHTTPTimeoutPolicy for EVERY responder,
only to make it easier later to tune timeout values
and/or to get feedback about which responder runs
into HTTP errors in debug output (especially time outs),
so that they can be tuned later. If we already understood
exactly what we were doing then most responders could
have been left alone and just return the default timeout
policy: by far most timeout policies are just a copy
of the default policy, currently.
This commit is not finished... It's a work in progress
(viewer runs fine with it though).
Comment fixes
Added some more debug support, not used yet (linux/libcwd only),
I used it, but won't commit the code that did.
Pass time_time parameter from post2() to prepRequest; only
used by mesh uploading at the moment.
CurlEasyHandle::mErrorBuffer (CURLOPT_ERRORBUFFER) can NOT be
used to retrieve information about an error returned by
curl_multi_info_read in CURLMsg::data::result. This buffer is
only initialized when a curl_easy_* call returns an error,
and those errors are already printed automagically.
Initialize the buffer with an empty string upon invokation
of an curl_easy_* call, so we are sure the error belongs to
the last call.
