Singularity-Viewer/SingularityViewer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'master' of https://github.com/AlericInglewood/SingularityViewer.git into testpit

Browse Source 
Conflicts:
	indra/newview/lldrawpoolavatar.cpp
	indra/newview/llviewertexturelist.cpp
	indra/newview/llworldmap.cpp
	indra/newview/pipeline.cpp
This commit is contained in:
Shyotl
2012-11-29 11:54:41 -06:00
parent 3a4b6cf6b6 e17405d80a
commit 794dbacea3
358 changed files with 27109 additions and 8865 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@@ -11,6 +11,7 @@
/bin-release
/indra/viewer-*
/indra/newview/vivox-runtime/
/indra/newview/dbghelp.dll
/libraries/
/lib/
*.pyc

114
LICENSES/FLOSS-exception.txt Normal file
View File

@@ -0,0 +1,114 @@
Linden Research, Inc. ("Linden Lab") Viewer FLOSS License Exception v0.5
The Linden Lab Exception for Free/Libre and Open Source Software-only
Applications Using Linden Lab Viewer Software (the "FLOSS Exception").
Exception Intent
Linden Lab is releasing the source code for certain software that
enables users to view or otherwise access the Second Life virtual
world environment (the "Viewer Software"), under version 2 of the GNU
General Public License (the "GPL"). The creation or distribution of
works based on the Program (as defined under the GPL) of the Viewer
Software may require the use of certain Free/Libre and Open Source
Software ("FLOSS") works that are subject to license agreements not
compatible with re-licensing under the GPL. Because we want to allow
the Viewer Software to be distributed with these FLOSS works, this
FLOSS Exception following exception applies subject to the terms and
conditions below.
Legal Terms and Conditions
As a special exception to the terms and conditions of version 2.0 of
the GPL:
You are free to distribute a work based on the Program that is formed
entirely from the Viewer Software (and any modifications thereof) and
one or more works that are independent and separate works not derived
from the Viewer Software, and are licensed under one or more of the
licenses listed below in section 1 (each, a "FLOSS Work") , as long
as:
A. You obey the GPL in all respects for the Viewer Software and any
work based on the Program, except for the FLOSS Works, for which
you must comply with B below,
B. all FLOSS Works,
i. are distributed subject to one of the FLOSS licenses
listed below, and
ii. the object code or executable form of the FLOSS Works are
accompanied by the complete corresponding
machine-readable source code for those FLOSS Works on the
same medium and under the same FLOSS license as the
corresponding object code or executable forms thereof,
and
C. any works that are aggregated with the Viewer Software or a work
based on the Program on a volume of a storage or distribution
medium in accordance with the GPL, and are not licensed under
the FLOSS licenses listed below, are independent and separate
works in themselves which are not derivatives of either the
Viewer Software, a work based on the Program or a FLOSS Work.
If the above conditions are not met, then the Viewer Software may only
be copied, modified, distributed or used under the terms and
conditions of the GPL or another valid licensing option from Linden
Lab.
1. FLOSS License List
License name Version(s)/Copyright Date
Academic Free License 2.0
Apache Software License 1.0/1.1/2.0
Apple Public Source License 2.0
Artistic license From Perl 5.8.0
BSD license "July 22 1999"
Common Development and
Distribution License (CDDL) 1.0
Common Public License 1.0
GNU Library or "Lesser" General
Public License (LGPL) 2.0/2.1
Jabber Open Source License 1.0
MIT License (As listed in file MIT-License.txt) -
Mozilla Public License (MPL) 1.0/1.1
Open Software License 2.0
OpenSSL license (with
original SSLeay license) "2003" ("1998")
PHP License 3.0
Python license (CNRI Python License) -
Python Software Foundation License 2.1.1
Sleepycat License "1999"
W3C License "2001"
X11 License "2001"
Zlib/libpng License -
Zope Public License 2.0
Due to the many variants of some of the above licenses, we require
that any variant of the above licenses be identical in substance to
the form approved by the Open Source Initiative and follow the 2003
version of the Free Software Foundation's Free Software Definition
(http://www.gnu.org/philosophy/free-sw.html) or version 1.9 of the
Open Source Definition by the Open Source Initiative
(http://www.opensource.org/docs/definition.php).
2. Definitions
Terms used, but not defined, herein shall have the meaning provided in
the GPL.
3. Applicability
This FLOSS Exception applies to all Viewer Software files that contain
a notice placed by Linden Lab saying that the Viewer Software may be
distributed under the terms of this FLOSS Exception. If you create or
distribute a work which is a work based on the Program for the Viewer
Software and any other work licensed under the GPL, then this FLOSS
Exception is not available for that work; thus, you must remove the
FLOSS Exception notice from that work and comply with the GPL in all
respects, including by retaining all GPL notices. You may choose to
redistribute a copy of the Viewer Software exclusively under the terms
of the GPL by removing the FLOSS Exception notice from that copy of
the Viewer Software, provided that the copy has never been modified by
you or any third party.

339
LICENSES/GPL-license.txt Normal file
View File

@@ -0,0 +1,339 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Lesser General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License.

36
LICENSES/LLQTWEBKIT_LICENSE.txt Normal file
View File

@@ -0,0 +1,36 @@
Source code
========
The license for the source code in this distribution should be clearly
marked on each source file. Unless otherwise specified, the source
code in this distribution ("Source Code") is provided by Linden Lab to
you under the terms of the GNU General Public License, version 2.0
("GPL"), unless you have obtained a separate licensing agreement
("Other License"), formally executed by you and Linden Lab. Terms of
the GPL can be found in GPL-license.txt in this distribution, or
online at http://secondlifegrid.net/technology-programs/license-virtual-world/viewerlicensing/gplv2
There are special exceptions to the terms and conditions of the GPL as
it is applied to this Source Code. View the full text of the exception
in the file FLOSS-exception.txt in this software distribution, or
online at http://secondlifegrid.net/technology-programs/license-virtual-world/viewerlicensing/flossexception
By copying, modifying or distributing this software, you acknowledge
that you have read and understood your obligations described above,
and agree to abide by those obligations.
ALL LINDEN LAB SOURCE CODE IS PROVIDED "AS IS." LINDEN LAB MAKES NO
WARRANTIES, EXPRESS, IMPLIED OR OTHERWISE, REGARDING ITS ACCURACY,
COMPLETENESS OR PERFORMANCE.
Logos and trademarks
==============
"Second Life" and "Linden Lab" are registered trademarks of Linden
Research, Inc. Other trademarks include (but are not limited to): the
names Linden and Linden Research, as well as the Linden Lab Hexagon
Design and the Second Life Hand Design logos.
Use of logos and trademarks are subject to the Linden Lab trademark
policy, available at:
http://secondlife.com/corporate/trademark/

27
README
View File

@@ -9,15 +9,30 @@
Sin-gu-la-ri-ty (noun) - a distinctive feature, a uniqueness; a point at which
continuity breaks up; a point in history at which machine becomes smarter than
humanity and/or fuses with it indivisively; or simply a cool sounding word with
my initials in it :)
the initials S.G. in it :)
Singularity Viewer is a Second Life protocol compatible client application. It
can be used to access Second LIfe service as well as a number of other such as
those based upon OpenSim plattform. It is directly based upon source code of
Ascent Viewer by Balseraph Software Group, which is in turn based upon source
code released by Linden Lab, with contributions from various sources.
Singularity Viewer is a SecondLife(tm) protocol compatible client application.
It can be used to access SecondLife services as well as a number of others such
as those based upon the OpenSim platform.
Singulariy is maintained by a small group of volunteers who can be contacted
both, in-world (SingularityViewer group) as well on IRC (#SingularityViewer
@ FreeNode). Bug requests and features requests can be submitted through our
Issue Tracket (http://code.google.com/p/singularity-viewer/issues/list or from
the viewer menu: Help --> Bug Reporting --> Singularity Issue Tracker...)
As this Readme grows out of date, please refer to
http://www.singularityviewer.org/about
00000000011111111112222222222333333333344444444445555555555666666666677777777778
12345678901234567890123456789012345678901234567890123456789012345678901234567890
History
The Singularity viewer was started by Siana Gearz in November 2010 by forking it
of the Ascent Viewer, by Balseraph Software Group, which in turn was based upon
source code released by Linden Lab.

1
doc/contributions.txt
View File

@@ -248,6 +248,7 @@ Celierra Darling
VWR-6975
Cron Stardust
VWR-10579
STORM-1919
Cypren Christenson
SNOW-129
SNOW-140

7
indra/CMakeLists.txt
View File

@@ -18,7 +18,7 @@ cmake_minimum_required(VERSION 2.6.2 FATAL_ERROR)
cmake_policy(SET CMP0003 OLD)
set(ROOT_PROJECT_NAME "Singularity" CACHE STRING
"The root project/makefile/solution name. Defaults to SecondLife.")
"The root project/makefile/solution name. Defaults to Singularity.")
project(${ROOT_PROJECT_NAME})
set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
@@ -46,6 +46,7 @@ endif(NOT STANDALONE)
add_custom_target(prepare DEPENDS ${prepare_depends})
add_subdirectory(cmake)
add_subdirectory(${LIBS_OPEN_PREFIX}aistatemachine)
add_subdirectory(${LIBS_OPEN_PREFIX}llaudio)
add_subdirectory(${LIBS_OPEN_PREFIX}llcharacter)
add_subdirectory(${LIBS_OPEN_PREFIX}llcommon)
@@ -61,6 +62,10 @@ add_subdirectory(${LIBS_OPEN_PREFIX}llvfs)
add_subdirectory(${LIBS_OPEN_PREFIX}llwindow)
add_subdirectory(${LIBS_OPEN_PREFIX}llxml)
if(STANDALONE)
add_subdirectory(${LIBS_OPEN_PREFIX}llqtwebkit)
endif(STANDALONE)
if (EXISTS ${LIBS_CLOSED_DIR}llkdu AND NOT STANDALONE)
add_subdirectory(${LIBS_CLOSED_PREFIX}llkdu)
endif (EXISTS ${LIBS_CLOSED_DIR}llkdu AND NOT STANDALONE)

39
indra/aistatemachine/CMakeLists.txt Normal file
View File

@@ -0,0 +1,39 @@
# -*- cmake -*-
project(aistatemachine)
include(00-Common)
include(LLCommon)
include(LLMessage)
include(LLMath)
include(LLXML)
include_directories(${CMAKE_CURRENT_SOURCE_DIR})
include_directories(
${LLCOMMON_INCLUDE_DIRS}
${LLMESSAGE_INCLUDE_DIRS}
${LLMATH_INCLUDE_DIRS}
${LLXML_INCLUDE_DIRS}
)
set(aistatemachine_SOURCE_FILES
aistatemachine.cpp
aitimer.cpp
)
set(aistatemachine_HEADER_FILES
CMakeLists.txt
aistatemachine.h
aitimer.h
)
set_source_files_properties(${aistatemachine_HEADER_FILES}
PROPERTIES HEADER_FILE_ONLY TRUE)
list(APPEND aistatemachine_SOURCE_FILES ${aistatemachine_HEADER_FILES})
add_library (aistatemachine ${aistatemachine_SOURCE_FILES})
add_dependencies(aistatemachine prepare)

92
indra/newview/statemachine/aistatemachine.cpp → indra/aistatemachine/aistatemachine.cpp
View File

@@ -32,16 +32,11 @@
#include <algorithm>
#include "llcallbacklist.h"
#include "llcontrol.h"
#include "llfasttimer.h"
#include "aithreadsafe.h"
#include "aistatemachine.h"
extern F64 calc_clock_frequency(void);
extern LLControlGroup gSavedSettings;
// Local variables.
namespace {
struct QueueElementComp;
@@ -66,28 +61,18 @@ namespace {
typedef std::vector<QueueElement> active_statemachines_type;
active_statemachines_type active_statemachines;
typedef std::vector<AIStateMachine*> continued_statemachines_type;
struct cscm_type
{
continued_statemachines_type continued_statemachines;
bool calling_mainloop;
};
AIThreadSafeDC<cscm_type> continued_statemachines_and_calling_mainloop;
}
// static
AIThreadSafeSimpleDC<U64> AIStateMachine::sMaxCount;
U64 AIStateMachine::sMaxCount;
AIThreadSafeDC<AIStateMachine::csme_type> AIStateMachine::sContinuedStateMachinesAndMainloopEnabled;
void AIStateMachine::updateSettings(void)
// static
void AIStateMachine::setMaxCount(F32 StateMachineMaxTime)
{
static const LLCachedControl<U32> StateMachineMaxTime("StateMachineMaxTime", 20);
static U32 last_StateMachineMaxTime = 0;
if (last_StateMachineMaxTime != StateMachineMaxTime)
{
Dout(dc::statemachine, "Initializing AIStateMachine::sMaxCount");
*AIAccess<U64>(sMaxCount) = calc_clock_frequency() * StateMachineMaxTime / 1000;
last_StateMachineMaxTime = StateMachineMaxTime;
}
llassert(is_main_thread());
Dout(dc::statemachine, "(Re)calculating AIStateMachine::sMaxCount");
sMaxCount = calc_clock_frequency() * StateMachineMaxTime / 1000;
}
//----------------------------------------------------------------------------
@@ -221,24 +206,23 @@ void AIStateMachine::locked_cont(void)
// If not_active is true then main-thread is not running this statemachine.
// It might call cont() (or set_state()) but never locked_cont(), and will never
// start actually running until we are done here and release the lock on
// continued_statemachines_and_calling_mainloop again. It is therefore safe
// sContinuedStateMachinesAndMainloopEnabled again. It is therefore safe
// to release mSetStateLock here, with as advantage that if we're not the main-
// thread and not_active is true, then the main-thread won't block when it has
// a timer running that times out and calls set_state().
mSetStateLock.unlock();
if (not_active)
{
AIWriteAccess<cscm_type> cscm_w(continued_statemachines_and_calling_mainloop);
AIWriteAccess<csme_type> csme_w(sContinuedStateMachinesAndMainloopEnabled);
// See above: it is not possible that mActive was changed since not_active
// was set to true above.
llassert_always(mActive == as_idle);
Dout(dc::statemachine, "Adding " << (void*)this << " to continued_statemachines");
cscm_w->continued_statemachines.push_back(this);
if (!cscm_w->calling_mainloop)
csme_w->continued_statemachines.push_back(this);
if (!csme_w->mainloop_enabled)
{
Dout(dc::statemachine, "Adding AIStateMachine::mainloop to gIdleCallbacks");
cscm_w->calling_mainloop = true;
gIdleCallbacks.addFunction(&AIStateMachine::mainloop);
Dout(dc::statemachine, "Activating AIStateMachine::mainloop.");
csme_w->mainloop_enabled = true;
}
mActive = as_queued;
llassert_always(!mIdle); // It should never happen that the main thread calls idle(), while another thread calls cont() concurrently.
@@ -499,11 +483,10 @@ void AIStateMachine::multiplex(U64 current_time)
}
//static
void AIStateMachine::add_continued_statemachines(void)
void AIStateMachine::add_continued_statemachines(AIReadAccess<csme_type>& csme_r)
{
AIReadAccess<cscm_type> cscm_r(continued_statemachines_and_calling_mainloop);
bool nonempty = false;
for (continued_statemachines_type::const_iterator iter = cscm_r->continued_statemachines.begin(); iter != cscm_r->continued_statemachines.end(); ++iter)
for (continued_statemachines_type::const_iterator iter = csme_r->continued_statemachines.begin(); iter != csme_r->continued_statemachines.end(); ++iter)
{
nonempty = true;
active_statemachines.push_back(QueueElement(*iter));
@@ -511,19 +494,15 @@ void AIStateMachine::add_continued_statemachines(void)
(*iter)->mActive = as_active;
}
if (nonempty)
AIWriteAccess<cscm_type>(cscm_r)->continued_statemachines.clear();
AIWriteAccess<csme_type>(csme_r)->continued_statemachines.clear();
}
static LLFastTimer::DeclareTimer FTM_STATEMACHINE("State Machine");
// static
void AIStateMachine::mainloop(void*)
void AIStateMachine::dowork(void)
{
LLFastTimer t(FTM_STATEMACHINE);
add_continued_statemachines();
llassert(!active_statemachines.empty());
// Run one or more state machines.
U64 total_clocks = 0;
U64 max_count = *AIAccess<U64>(sMaxCount);
for (active_statemachines_type::iterator iter = active_statemachines.begin(); iter != active_statemachines.end(); ++iter)
{
AIStateMachine& statemachine(iter->statemachine());
@@ -537,7 +516,7 @@ void AIStateMachine::mainloop(void*)
U64 delta = get_clock_count() - start;
iter->add(delta);
total_clocks += delta;
if (total_clocks >= max_count)
if (total_clocks >= sMaxCount)
{
#ifndef LL_RELEASE_FOR_DOWNLOAD
llwarns << "AIStateMachine::mainloop did run for " << (total_clocks * 1000 / calc_clock_frequency()) << " ms." << llendl;
@@ -590,12 +569,11 @@ void AIStateMachine::mainloop(void*)
if (active_statemachines.empty())
{
// If this was the last state machine, remove mainloop from the IdleCallbacks.
AIReadAccess<cscm_type> cscm_r(continued_statemachines_and_calling_mainloop);
if (cscm_r->continued_statemachines.empty() && cscm_r->calling_mainloop)
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled, true);
if (csme_r->continued_statemachines.empty() && csme_r->mainloop_enabled)
{
Dout(dc::statemachine, "Removing AIStateMachine::mainloop from gIdleCallbacks");
AIWriteAccess<cscm_type>(cscm_r)->calling_mainloop = false;
gIdleCallbacks.deleteFunction(&AIStateMachine::mainloop);
Dout(dc::statemachine, "Deactivating AIStateMachine::mainloop: no active state machines left.");
AIWriteAccess<csme_type>(csme_r)->mainloop_enabled = false;
}
}
}
@@ -604,14 +582,17 @@ void AIStateMachine::mainloop(void*)
void AIStateMachine::flush(void)
{
DoutEntering(dc::curl, "AIStateMachine::flush(void)");
add_continued_statemachines();
{
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled);
add_continued_statemachines(csme_r);
}
// Abort all state machines.
for (active_statemachines_type::iterator iter = active_statemachines.begin(); iter != active_statemachines.end(); ++iter)
{
AIStateMachine& statemachine(iter->statemachine());
if (statemachine.abortable())
{
// We can't safely call abort() here for non-running (run() was called, but they we're initialized yet) statemachines,
// We can't safely call abort() here for non-running (run() was called, but they weren't initialized yet) statemachines,
// because that might call kill() which in some cases is undesirable (ie, when it is owned by a partent that will
// also call abort() on it when it is aborted itself).
if (statemachine.running())
@@ -626,21 +607,22 @@ void AIStateMachine::flush(void)
for(;;)
{
{
AIReadAccess<cscm_type> cscm_r(continued_statemachines_and_calling_mainloop);
if (!cscm_r->calling_mainloop)
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled);
if (!csme_r->mainloop_enabled)
break;
}
mainloop(NULL);
mainloop();
}
if (batch == 1)
break;
add_continued_statemachines();
// Kill all state machines.
for (active_statemachines_type::iterator iter = active_statemachines.begin(); iter != active_statemachines.end(); ++iter)
{
AIStateMachine& statemachine(iter->statemachine());
if (statemachine.running())
statemachine.kill();
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled);
add_continued_statemachines(csme_r);
}
}
// At this point all statemachines should be idle.
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled);
llinfos << "Current number of continued statemachines: " << csme_r->continued_statemachines.size() << llendl;
llinfos << "Current number of active statemachines: " << active_statemachines.size() << llendl;
llassert(csme_r->continued_statemachines.empty() && active_statemachines.empty());
}

33
indra/newview/statemachine/aistatemachine.h → indra/aistatemachine/aistatemachine.h
View File

@@ -192,6 +192,15 @@ class AIStateMachine {
as_active // State machine is on active_statemachines list.
};
//! Type of continued_statemachines.
typedef std::vector<AIStateMachine*> continued_statemachines_type;
//! Type of sContinuedStateMachinesAndMainloopEnabled.
struct csme_type
{
continued_statemachines_type continued_statemachines;
bool mainloop_enabled;
};
public:
typedef U32 state_type; //!< The type of mRunState
@@ -230,7 +239,8 @@ class AIStateMachine {
};
callback_type* mCallback; //!< Pointer to signal/connection, or NULL when not connected.
static AIThreadSafeSimpleDC<U64> sMaxCount; //!< Number of cpu clocks below which we start a new state machine within the same frame.
static U64 sMaxCount; //!< Number of cpu clocks below which we start a new state machine within the same frame.
static AIThreadSafeDC<csme_type> sContinuedStateMachinesAndMainloopEnabled; //!< Read/write locked variable pair.
protected:
LLMutex mSetStateLock; //!< For critical areas in set_state() and locked_cont().
@@ -244,7 +254,7 @@ class AIStateMachine {
#ifdef SHOW_ASSERT
, mContThread(AIThreadID::none), mCalledThreadUnsafeIdle(false)
#endif
{ updateSettings(); }
{ }
protected:
//! The user should call 'kill()', not delete a AIStateMachine (derived) directly.
@@ -339,7 +349,7 @@ class AIStateMachine {
// Other.
//! Called whenever the StateMachineMaxTime setting is changed.
static void updateSettings(void);
static void setMaxCount(F32 StateMachineMaxTime);
//---------------------------------------
// Accessors.
@@ -365,11 +375,24 @@ class AIStateMachine {
char const* state_str(state_type state);
private:
static void add_continued_statemachines(void);
static void mainloop(void*);
static void add_continued_statemachines(AIReadAccess<csme_type>& csme_r);
static void dowork(void);
void multiplex(U64 current_time);
public:
//! Call this once per frame to give the statemachines CPU cycles.
static void mainloop(void)
{
{
AIReadAccess<csme_type> csme_r(sContinuedStateMachinesAndMainloopEnabled, true);
if (!csme_r->mainloop_enabled)
return;
if (!csme_r->continued_statemachines.empty())
add_continued_statemachines(csme_r);
}
dowork();
}
//! Abort all running state machines and then run mainloop until all state machines are idle (called when application is exiting).
static void flush(void);

0
indra/newview/statemachine/aitimer.cpp → indra/aistatemachine/aitimer.cpp
View File

0
indra/newview/statemachine/aitimer.h → indra/aistatemachine/aitimer.h
View File

4
indra/cmake/AIStateMachine.cmake
View File

@@ -1,4 +1,4 @@
# -*- cmake -*-
set(AISTATEMACHINE_INCLUDE_DIRS statemachine)
set(AISTATEMACHINE_LIBRARIES statemachine)
set(AISTATEMACHINE_INCLUDE_DIRS ${LIBS_OPEN_DIR}/aistatemachine)
set(AISTATEMACHINE_LIBRARIES aistatemachine)

2
indra/cmake/CMakeLists.txt
View File

@@ -54,6 +54,7 @@ set(cmake_SOURCE_FILES
LLMessage.cmake
LLPlugin.cmake
LLPrimitive.cmake
LLQtWebkit.cmake
LLRender.cmake
LLScene.cmake
LLUI.cmake
@@ -70,6 +71,7 @@ set(cmake_SOURCE_FILES
PNG.cmake
Python.cmake
Prebuilt.cmake
Qt4.cmake
RunBuildTest.cmake
TemplateCheck.cmake
Tut.cmake

4
indra/cmake/LLMessage.cmake
View File

@@ -4,12 +4,14 @@ include(CARes)
include(CURL)
include(OpenSSL)
include(XmlRpcEpi)
include(AIStateMachine)
set(LLMESSAGE_INCLUDE_DIRS
${LIBS_OPEN_DIR}/llmessage
${CARES_INCLUDE_DIRS}
${CURL_INCLUDE_DIRS}
${OPENSSL_INCLUDE_DIRS}
${AISTATEMACHINE_INCLUDE_DIRS}
)
set(LLMESSAGE_LIBRARIES llmessage)
set(LLMESSAGE_LIBRARIES llmessage aistatemachine)

11
indra/cmake/LLQtWebkit.cmake Normal file
View File

@@ -0,0 +1,11 @@
# -*- cmake -*-
if (STANDALONE)
set(LLQTWEBKIT_INCLUDE_DIR
${LIBS_OPEN_DIR}/llqtwebkit
)
set(LLQTWEBKIT_LIBRARY
llqtwebkit
)
endif (STANDALONE)

12
indra/cmake/Qt4.cmake Normal file
View File

@@ -0,0 +1,12 @@
# -*- cmake -*-
include(Prebuilt)
if (STANDALONE)
set(Qt4_FIND_REQUIRED ON)
include(FindQt4)
find_package(Qt4 4.7.0 COMPONENTS QtCore QtGui QtNetwork QtOpenGL QtWebKit REQUIRED)
include(${QT_USE_FILE})
add_definitions(${QT_DEFINITIONS})
endif (STANDALONE)

4
indra/cmake/StateMachine.cmake Normal file
View File

@@ -0,0 +1,4 @@
# -*- cmake -*-
set(STATEMACHINE_INCLUDE_DIRS statemachine)
set(STATEMACHINE_LIBRARIES statemachine)

5
indra/cmake/ViewerMiscLibs.cmake
View File

@@ -19,8 +19,3 @@ else (NOT STANDALONE)
endif(LINUX AND ${ARCH} STREQUAL "x86_64")
set(STANDALONE ON)
endif(NOT STANDALONE)
if (WINDOWS)
use_prebuilt_binary(dbghelp)
endif (WINDOWS)

15
indra/cmake/WebKitLibPlugin.cmake
View File

@@ -1,19 +1,10 @@
# -*- cmake -*-
include(Linking)
include(Prebuilt)
include(LLQtWebkit)
include(Qt4)
if (STANDALONE)
# The minimal version, 4.4.3, is rather arbitrary: it's the version in Debian/Lenny.
find_package(Qt4 4.4.3 COMPONENTS QtCore QtGui QtNetwork QtOpenGL QtWebKit REQUIRED)
include(${QT_USE_FILE})
set(QTDIR $ENV{QTDIR})
if (QTDIR AND NOT "${QT_BINARY_DIR}" STREQUAL "${QTDIR}/bin")
message(FATAL_ERROR "\"${QT_BINARY_DIR}\" is unequal \"${QTDIR}/bin\"; "
"Qt is found by looking for qmake in your PATH. "
"Please set your PATH such that 'qmake' is found in \$QTDIR/bin, "
"or unset QTDIR if the found Qt is correct.")
endif (QTDIR AND NOT "${QT_BINARY_DIR}" STREQUAL "${QTDIR}/bin")
find_package(LLQtWebkit REQUIRED QUIET)
set(WEBKITLIBPLUGIN OFF CACHE BOOL
"WEBKITLIBPLUGIN support for the llplugin/llmedia test apps.")
else (STANDALONE)
@@ -46,7 +37,7 @@ elseif (DARWIN)
)
elseif (LINUX)
if (STANDALONE)
set(WEBKIT_PLUGIN_LIBRARIES ${LLQTWEBKIT_LIBRARY} ${QT_LIBRARIES})
set(WEBKIT_PLUGIN_LIBRARIES ${LLQTWEBKIT_LIBRARY} ${QT_LIBRARIES} ${QT_PLUGIN_LIBRARIES})
else (STANDALONE)
set(WEBKIT_PLUGIN_LIBRARIES
llqtwebkit

63
indra/cwdebug/debug.cc
View File

@@ -49,6 +49,8 @@
namespace debug {
#if CWDEBUG_LOCATION
ll_thread_local size_t BackTrace::S_number;
void BackTrace::dump_backtrace(void) const
{
for (int frame = 0; frame < frames(); ++frame)
@@ -67,6 +69,67 @@ void BackTrace::dump_backtrace(void) const
Dout(dc::finish, mangled_function_name);
}
}
void BackTraces::store_trace(size_t trace)
{
mBackTraces.push_back(trace);
}
void BackTraces::remove_trace(size_t trace)
{
trace_container_type::iterator iter = mBackTraces.begin();
while (iter != mBackTraces.end())
{
if (*iter == trace)
{
*iter = mBackTraces.back();
mBackTraces.pop_back();
return;
}
++iter;
}
DoutFatal(dc::core, "Trace doesn't exist!");
}
void BackTraces::dump(void) const
{
Dout(dc::backtrace|continued_cf, "Dump for (BackTraces*)" << (void*)this << " (" << mBackTraces.size() << " backtraces): ");
for (trace_container_type::const_iterator iter = mBackTraces.begin(); iter != mBackTraces.end(); ++iter)
{
Dout(dc::continued|nonewline_cf, *iter << ' ');
}
Dout(dc::finish, "");
}
BackTraceTracker::BackTraceTracker(BackTraces* back_traces) : mBackTraces(back_traces)
{
BACKTRACE;
mTrace = BackTrace::S_number;
mBackTraces->store_trace(mTrace);
}
BackTraceTracker::~BackTraceTracker()
{
mBackTraces->remove_trace(mTrace);
}
BackTraceTracker::BackTraceTracker(BackTraceTracker const& orig) : mBackTraces(orig.mBackTraces)
{
BACKTRACE;
mTrace = BackTrace::S_number;
mBackTraces->store_trace(mTrace);
}
BackTraceTracker& BackTraceTracker::operator=(BackTraceTracker const& orig)
{
mBackTraces->remove_trace(mTrace);
mBackTraces = orig.mBackTraces;
BACKTRACE;
mTrace = BackTrace::S_number;
mBackTraces->store_trace(mTrace);
return *this;
}
#endif // CWDEBUG_LOCATION
#if CWDEBUG_ALLOC && CWDEBUG_LOCATION

72
indra/cwdebug/debug.h
View File

@@ -173,7 +173,9 @@ extern LL_COMMON_API fake_channel const notice;
#include <boost/shared_array.hpp>
#if CWDEBUG_LOCATION
#include <execinfo.h> // Needed for 'backtrace'.
#include "llpreprocessor.h"
#endif
#include <set>
#define CWD_API __attribute__ ((visibility("default")))
@@ -273,6 +275,8 @@ class BackTrace {
private:
boost::shared_array<void*> M_buffer;
int M_frames;
public:
static ll_thread_local size_t S_number;
public:
BackTrace(void** buffer, int frames) : M_buffer(new void* [frames]), M_frames(frames) { std::memcpy(M_buffer.get(), buffer, sizeof(void*) * frames); }
@@ -299,19 +303,81 @@ extern pthread_mutex_t backtrace_mutex;
using namespace debug; \
void* buffer[32]; \
int frames = backtrace(buffer, 32); \
size_t size; \
{ \
pthread_mutex_lock(&backtrace_mutex); \
backtraces.push_back(BackTrace(buffer, frames)); \
size = backtraces.size(); \
BackTrace::S_number = backtraces.size(); \
pthread_mutex_unlock(&backtrace_mutex); \
} \
Dout(dc::backtrace, "Stored backtrace #" << size); \
Dout(dc::backtrace, "Stored backtrace #" << BackTrace::S_number); \
} while(0)
class LL_COMMON_API BackTraces {
private:
typedef std::vector<size_t> trace_container_type;
trace_container_type mBackTraces;
public:
void store_trace(size_t trace);
void remove_trace(size_t trace);
void dump(void) const;
};
class LL_COMMON_API BackTraceTracker {
private:
BackTraces* mBackTraces;
size_t mTrace;
public:
BackTraceTracker(BackTraces* back_traces);
~BackTraceTracker();
BackTraceTracker(BackTraceTracker const&);
BackTraceTracker& operator=(BackTraceTracker const&);
void dump(void) const { mBackTraces->dump(); }
};
#else
#define BACKTRACE do { } while(0)
#endif // CWDEBUG_LOCATION
template<class T>
class LL_COMMON_API InstanceTracker {
private:
T const* mInstance;
static pthread_mutex_t sInstancesMutex;
static std::set<T const*> sInstances;
static void remember(T const* instance) { pthread_mutex_lock(&sInstancesMutex); sInstances.insert(instance); pthread_mutex_unlock(&sInstancesMutex); }
static void forget(T const* instance) { pthread_mutex_lock(&sInstancesMutex); sInstances.erase(instance); pthread_mutex_unlock(&sInstancesMutex); }
public:
InstanceTracker(T const* instance) : mInstance(instance) { remember(mInstance); }
~InstanceTracker() { forget(mInstance); }
InstanceTracker& operator=(InstanceTracker const& orig) { forget(mInstance); mInstance = orig.mInstance; remember(mInstance); return *this; }
static void dump(void);
private:
// Non-copyable. Instead of copying, call InstanceTracker(T const*) with the this pointer of the new instance.
InstanceTracker(InstanceTracker const& orig);
};
template<class T>
pthread_mutex_t InstanceTracker<T>::sInstancesMutex = PTHREAD_MUTEX_INITIALIZER;
template<class T>
std::set<T const*> InstanceTracker<T>::sInstances;
template<class T>
void InstanceTracker<T>::dump(void)
{
pthread_mutex_lock(&sInstancesMutex);
for (typename std::set<T const*>::iterator iter = sInstances.begin(); iter != sInstances.end(); ++iter)
{
std::cout << *iter << std::endl;
}
pthread_mutex_unlock(&sInstancesMutex);
}
} // namespace debug
//! Debugging macro.

11
indra/llcharacter/llcharacter.cpp
View File

@@ -193,21 +193,30 @@ void LLCharacter::requestStopMotion( LLMotion* motion)
//-----------------------------------------------------------------------------
// updateMotions()
//-----------------------------------------------------------------------------
static LLFastTimer::DeclareTimer FTM_UPDATE_ANIMATION("Update Animation");
static LLFastTimer::DeclareTimer FTM_UPDATE_HIDDEN_ANIMATION("Update Hidden Anim");
static LLFastTimer::DeclareTimer FTM_UPDATE_MOTIONS("Update Motions");
void LLCharacter::updateMotions(e_update_t update_type)
{
if (update_type == HIDDEN_UPDATE)
{
LLFastTimer t(FTM_UPDATE_HIDDEN_ANIMATION);
mMotionController.updateMotionsMinimal();
}
else
{
LLFastTimer t(FTM_UPDATE_ANIMATION);
// unpause if the number of outstanding pause requests has dropped to the initial one
if (mMotionController.isPaused() && mPauseRequest->getNumRefs() == 1)
{
mMotionController.unpauseAllMotions();
}
bool force_update = (update_type == FORCE_UPDATE);
mMotionController.updateMotions(force_update);
{
LLFastTimer t(FTM_UPDATE_MOTIONS);
mMotionController.updateMotions(force_update);
}
}
}

7
indra/llcharacter/llmotioncontroller.cpp
View File

@@ -547,6 +547,8 @@ void LLMotionController::updateIdleActiveMotions()
//-----------------------------------------------------------------------------
// updateMotionsByType()
//-----------------------------------------------------------------------------
static LLFastTimer::DeclareTimer FTM_MOTION_ON_UPDATE("Motion onUpdate");
void LLMotionController::updateMotionsByType(LLMotion::LLMotionBlendType anim_type)
{
BOOL update_result = TRUE;
@@ -704,7 +706,10 @@ void LLMotionController::updateMotionsByType(LLMotion::LLMotionBlendType anim_ty
}
// perform motion update
update_result = motionp->onUpdate(mAnimTime - motionp->mActivationTimestamp, last_joint_signature);
{
LLFastTimer t(FTM_MOTION_ON_UPDATE);
update_result = motionp->onUpdate(mAnimTime - motionp->mActivationTimestamp, last_joint_signature);
}
}
//**********************

6
indra/llcommon/aithreadsafe.h
View File

@@ -329,14 +329,14 @@ struct AIReadAccessConst
};
//! Construct a AIReadAccessConst from a constant AIThreadSafe.
AIReadAccessConst(AIThreadSafe<T> const& wrapper)
AIReadAccessConst(AIThreadSafe<T> const& wrapper, bool high_priority = false)
: mWrapper(const_cast<AIThreadSafe<T>&>(wrapper)),
mState(readlocked)
#if AI_NEED_ACCESS_CC
, mIsCopyConstructed(false)
#endif
{
mWrapper.mRWLock.rdlock();
mWrapper.mRWLock.rdlock(high_priority);
}
//! Destruct the AI*Access object.
@@ -393,7 +393,7 @@ struct AIReadAccess : public AIReadAccessConst<T>
using AIReadAccessConst<T>::readlocked;
//! Construct a AIReadAccess from a non-constant AIThreadSafe.
AIReadAccess(AIThreadSafe<T>& wrapper) : AIReadAccessConst<T>(wrapper, readlocked) { this->mWrapper.mRWLock.rdlock(); }
AIReadAccess(AIThreadSafe<T>& wrapper, bool high_priority = false) : AIReadAccessConst<T>(wrapper, readlocked) { this->mWrapper.mRWLock.rdlock(high_priority); }
protected:
//! Constructor used by AIWriteAccess.

35
indra/llcommon/llapr.cpp
View File

@@ -36,26 +36,39 @@
#include "llapr.h"
#include "llscopedvolatileaprpool.h"
LLFastTimer::DeclareTimer FT_WAIT_FOR_SCOPEDLOCK("LLScopedLock");
//---------------------------------------------------------------------
//
// LLScopedLock
//
LLScopedLock::LLScopedLock(apr_thread_mutex_t* mutex) : mMutex(mutex)
{
if(mutex)
mLocked = !!mutex;
if (LL_LIKELY(mutex))
{
if(ll_apr_warn_status(apr_thread_mutex_lock(mMutex)))
apr_status_t status = apr_thread_mutex_trylock(mMutex);
while (LL_UNLIKELY(status != APR_SUCCESS))
{
mLocked = false;
if (APR_STATUS_IS_EBUSY(status))
{
if (AIThreadID::in_main_thread_inline())
{
LLFastTimer ft1(FT_WAIT_FOR_SCOPEDLOCK);
status = apr_thread_mutex_lock(mMutex);
}
else
{
status = apr_thread_mutex_lock(mMutex);
}
}
else
{
ll_apr_warn_status(status);
mLocked = false;
return;
}
}
else
{
mLocked = true;
}
}
else
{
mLocked = false;
}
}

2
indra/llcommon/llfile.h
View File

@@ -227,7 +227,7 @@ public:
#endif
/**
* @breif filesize helpers.
* @brief filesize helpers.
*
* The file size helpers are not considered particularly efficient,
* and should only be used for config files and the like -- not in a

1
indra/llcommon/llmemtype.cpp
View File

@@ -179,7 +179,6 @@ LLMemType::DeclareMemType LLMemType::MTYPE_IO_BUFFER("IoBuffer");
LLMemType::DeclareMemType LLMemType::MTYPE_IO_HTTP_SERVER("IoHttpServer");
LLMemType::DeclareMemType LLMemType::MTYPE_IO_SD_SERVER("IoSDServer");
LLMemType::DeclareMemType LLMemType::MTYPE_IO_SD_CLIENT("IoSDClient");
LLMemType::DeclareMemType LLMemType::MTYPE_IO_URL_REQUEST("IOUrlRequest");
LLMemType::DeclareMemType LLMemType::MTYPE_DIRECTX_INIT("DirectXInit");

1
indra/llcommon/llmemtype.h
View File

@@ -223,7 +223,6 @@ public:
static DeclareMemType MTYPE_IO_HTTP_SERVER;
static DeclareMemType MTYPE_IO_SD_SERVER;
static DeclareMemType MTYPE_IO_SD_CLIENT;
static DeclareMemType MTYPE_IO_URL_REQUEST;
static DeclareMemType MTYPE_DIRECTX_INIT;

14
indra/llcommon/llstring.h
View File

@@ -368,6 +368,7 @@ public:
* should work.
*/
static void _makeASCII(string_type& string);
static bool _isASCII(std::basic_string<T> const& string);
// Conversion to other data types
static BOOL convertToBOOL(const string_type& string, BOOL& value);
@@ -1473,6 +1474,19 @@ void LLStringUtilBase<T>::_makeASCII(string_type& string)
}
}
template<class T>
bool LLStringUtilBase<T>::_isASCII(std::basic_string<T> const& string)
{
size_type const len = string.length();
T bit_collector = 0;
for (size_type i = 0; i < len; ++i)
{
bit_collector |= string[i];
}
T const ascii_bits = 0x7f;
return !(bit_collector & ~ascii_bits);
}
// static
template<class T>
void LLStringUtilBase<T>::copy( T* dst, const T* src, size_type dst_size )

34
indra/llcommon/llthread.cpp
View File

@@ -113,12 +113,11 @@ void *APR_THREAD_FUNC LLThread::staticRun(apr_thread_t *apr_threadp, void *datap
// Only now print this info [doing that before setting mStatus
// to STOPPED makes it much more likely that another thread runs
// after the LLCurl::Multi::run() function exits and we actually
// change this variable (which really SHOULD have been inside
// the critical area of the mSignal lock)].
// after the AICurlPrivate::curlthread::AICurlThread::run() function
// exits and we actually change this variable (which really SHOULD
// have been inside the critical area of the mSignal lock)].
lldebugs << "LLThread::staticRun() Exiting: " << name << llendl;
--sRunning; // Would be better to do this after joining with the thread, but we don't join :/
return NULL;
}
@@ -383,9 +382,19 @@ LLCondition::~LLCondition()
mAPRCondp = NULL;
}
LLFastTimer::DeclareTimer FT_WAIT_FOR_CONDITION("LLCondition::wait()");
void LLCondition::wait()
{
apr_thread_cond_wait(mAPRCondp, mAPRMutexp);
if (AIThreadID::in_main_thread_inline())
{
LLFastTimer ft1(FT_WAIT_FOR_CONDITION);
apr_thread_cond_wait(mAPRCondp, mAPRMutexp);
}
else
{
apr_thread_cond_wait(mAPRCondp, mAPRMutexp);
}
}
void LLCondition::signal()
@@ -410,6 +419,8 @@ bool LLMutexBase::isSelfLocked() const
return mLockingThread.equals_current_thread_inline();
}
LLFastTimer::DeclareTimer FT_WAIT_FOR_MUTEX("LLMutexBase::lock()");
void LLMutexBase::lock()
{
if (mLockingThread.equals_current_thread_inline())
@@ -418,7 +429,18 @@ void LLMutexBase::lock()
return;
}
apr_thread_mutex_lock(mAPRMutexp);
if (APR_STATUS_IS_EBUSY(apr_thread_mutex_trylock(mAPRMutexp)))
{
if (AIThreadID::in_main_thread_inline())
{
LLFastTimer ft1(FT_WAIT_FOR_MUTEX);
apr_thread_mutex_lock(mAPRMutexp);
}
else
{
apr_thread_mutex_lock(mAPRMutexp);
}
}
mLockingThread.reset_inline();
}

5
indra/llcommon/lltimer.cpp
View File

@@ -262,6 +262,11 @@ U64 totalTime()
// No wrapping, we're all okay.
gTotalTimeClockCount += current_clock_count - gLastTotalTimeClockCount;
}
else if((gLastTotalTimeClockCount - current_clock_count)<0xFFFF)
{
//clock is glitching and walking backwards - ignore it
gTotalTimeClockCount = gLastTotalTimeClockCount;
}
else
{
// We've wrapped. Compensate correctly

18
indra/llcrashlogger/llcrashlogger.cpp
View File

@@ -48,14 +48,20 @@
#include "llpumpio.h"
#include "llhttpclient.h"
#include "llsdserialize.h"
#include "llcurl.h"
LLPumpIO* gServicePump;
BOOL gBreak = false;
BOOL gSent = false;
class LLCrashLoggerResponder : public LLHTTPClient::Responder
class AIHTTPTimeoutPolicy;
extern AIHTTPTimeoutPolicy crashLoggerResponder_timeout;
class LLCrashLoggerResponder : public LLHTTPClient::ResponderWithResult
{
public:
virtual AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const { return crashLoggerResponder_timeout; }
LLCrashLoggerResponder()
{
}
@@ -308,14 +314,14 @@ bool LLCrashLogger::saveCrashBehaviorSetting(S32 crash_behavior)
return true;
}
bool LLCrashLogger::runCrashLogPost(std::string host, LLSD data, std::string msg, int retries, int timeout)
bool LLCrashLogger::runCrashLogPost(std::string host, LLSD data, std::string msg, int retries)
{
gBreak = false;
std::string status_message;
for(int i = 0; i < retries; ++i)
{
status_message = llformat("%s, try %d...", msg.c_str(), i+1);
LLHTTPClient::post(host, data, new LLCrashLoggerResponder(), timeout);
LLHTTPClient::post(host, data, new LLCrashLoggerResponder);
while(!gBreak)
{
updateApplication(status_message);
@@ -350,12 +356,12 @@ bool LLCrashLogger::sendCrashLogs()
// *TODO: Translate
if(mCrashHost != "")
{
sent = runCrashLogPost(mCrashHost, post_data, std::string("Sending to server"), 3, 5);
sent = runCrashLogPost(mCrashHost, post_data, std::string("Sending to server"), 3);
}
if(!sent)
{
sent = runCrashLogPost(mAltCrashHost, post_data, std::string("Sending to alternate server"), 3, 5);
sent = runCrashLogPost(mAltCrashHost, post_data, std::string("Sending to alternate server"), 3);
}
mSentCrashLogs = sent;
@@ -367,6 +373,7 @@ void LLCrashLogger::updateApplication(const std::string& message)
{
gServicePump->pump();
gServicePump->callback();
//FIXME: AIStateMachine::mainloop(); needs CPU cycles. Can't call it from here though, because it uses gSavedSettings which is part of newview.
}
bool LLCrashLogger::init()
@@ -394,7 +401,6 @@ bool LLCrashLogger::init()
}
gServicePump = new LLPumpIO;
LLHTTPClient::setPump(*gServicePump);
//If we've opened the crash logger, assume we can delete the marker file if it exists
if( gDirUtilp )

4
indra/llcrashlogger/llcrashlogger.h
View File

@@ -40,6 +40,8 @@
#include "llsd.h"
#include "llcontrol.h"
class AIHTTPTimeoutPolicy;
class LLCrashLogger : public LLApp
{
public:
@@ -57,7 +59,7 @@ public:
virtual bool cleanup() { return true; }
void setUserText(const std::string& text) { mCrashInfo["UserNotes"] = text; }
S32 getCrashBehavior() { return mCrashBehavior; }
bool runCrashLogPost(std::string host, LLSD data, std::string msg, int retries, int timeout);
bool runCrashLogPost(std::string host, LLSD data, std::string msg, int retries);
protected:
S32 mCrashBehavior;
BOOL mCrashInPreviousExec;

4
indra/llimage/CMakeLists.txt
View File

@@ -22,13 +22,11 @@ include_directories(
)
set(llimage_SOURCE_FILES
aes.cpp
llimagebmp.cpp
llimage.cpp
llimagedxt.cpp
llimagej2c.cpp
llimagejpeg.cpp
llimagemetadatareader.cpp
llimagepng.cpp
llimagetga.cpp
llimageworker.cpp
@@ -37,13 +35,11 @@ set(llimage_SOURCE_FILES
set(llimage_HEADER_FILES
CMakeLists.txt
aes.h
llimage.h
llimagebmp.h
llimagedxt.h
llimagej2c.h
llimagejpeg.h
llimagemetadatareader.h
llimagepng.h
llimagetga.h
llimageworker.h

1380
indra/llimage/aes.cpp
View File

File diff suppressed because it is too large Load Diff

190
indra/llimage/aes.h
View File

@@ -1,190 +0,0 @@
//Rijndael.h
#ifndef __RIJNDAEL_H__
#define __RIJNDAEL_H__
#include <exception>
#include <string>
#include <cstring>
using namespace std;
//Rijndael (pronounced Reindaal) is a block cipher, designed by Joan Daemen and Vincent Rijmen as a candidate algorithm for the AES.
//The cipher has a variable block length and key length. The authors currently specify how to use keys with a length
//of 128, 192, or 256 bits to encrypt blocks with al length of 128, 192 or 256 bits (all nine combinations of
//key length and block length are possible). Both block length and key length can be extended very easily to
// multiples of 32 bits.
//Rijndael can be implemented very efficiently on a wide range of processors and in hardware.
//This implementation is based on the Java Implementation used with the Cryptix toolkit found at:
//http://www.esat.kuleuven.ac.be/~rijmen/rijndael/rijndael.zip
//Java code authors: Raif S. Naffah, Paulo S. L. M. Barreto
//This Implementation was tested against KAT test published by the authors of the method and the
//results were identical.
class CRijndael
{
public:
//Operation Modes
//The Electronic Code Book (ECB), Cipher Block Chaining (CBC) and Cipher Feedback Block (CFB) modes
//are implemented.
//In ECB mode if the same block is encrypted twice with the same key, the resulting
//ciphertext blocks are the same.
//In CBC Mode a ciphertext block is obtained by first xoring the
//plaintext block with the previous ciphertext block, and encrypting the resulting value.
//In CFB mode a ciphertext block is obtained by encrypting the previous ciphertext block
//and xoring the resulting value with the plaintext.
enum { ECB=0, CBC=1, CFB=2 };
private:
enum { DEFAULT_BLOCK_SIZE=16 };
enum { MAX_BLOCK_SIZE=32, MAX_ROUNDS=14, MAX_KC=8, MAX_BC=8 };
//Auxiliary Functions
//Multiply two elements of GF(2^m)
static int Mul(int a, int b)
{
return (a != 0 && b != 0) ? sm_alog[(sm_log[a & 0xFF] + sm_log[b & 0xFF]) % 255] : 0;
}
//Convenience method used in generating Transposition Boxes
static int Mul4(int a, char b[])
{
if(a == 0)
return 0;
a = sm_log[a & 0xFF];
int a0 = (b[0] != 0) ? sm_alog[(a + sm_log[b[0] & 0xFF]) % 255] & 0xFF : 0;
int a1 = (b[1] != 0) ? sm_alog[(a + sm_log[b[1] & 0xFF]) % 255] & 0xFF : 0;
int a2 = (b[2] != 0) ? sm_alog[(a + sm_log[b[2] & 0xFF]) % 255] & 0xFF : 0;
int a3 = (b[3] != 0) ? sm_alog[(a + sm_log[b[3] & 0xFF]) % 255] & 0xFF : 0;
return a0 << 24 | a1 << 16 | a2 << 8 | a3;
}
public:
//CONSTRUCTOR
CRijndael();
//DESTRUCTOR
virtual ~CRijndael();
//Expand a user-supplied key material into a session key.
// key - The 128/192/256-bit user-key to use.
// chain - initial chain block for CBC and CFB modes.
// keylength - 16, 24 or 32 bytes
// blockSize - The block size in bytes of this Rijndael (16, 24 or 32 bytes).
void MakeKey(char const* key, char const* chain, int keylength=DEFAULT_BLOCK_SIZE, int blockSize=DEFAULT_BLOCK_SIZE);
private:
//Auxiliary Function
void Xor(char* buff, char const* chain)
{
if(false==m_bKeyInit)
throw std::string(sm_szErrorMsg1);
for(int i=0; i<m_blockSize; i++)
*(buff++) ^= *(chain++);
}
//Convenience method to encrypt exactly one block of plaintext, assuming
//Rijndael's default block size (128-bit).
// in - The plaintext
// result - The ciphertext generated from a plaintext using the key
void DefEncryptBlock(char const* in, char* result);
//Convenience method to decrypt exactly one block of plaintext, assuming
//Rijndael's default block size (128-bit).
// in - The ciphertext.
// result - The plaintext generated from a ciphertext using the session key.
void DefDecryptBlock(char const* in, char* result);
public:
//Encrypt exactly one block of plaintext.
// in - The plaintext.
// result - The ciphertext generated from a plaintext using the key.
void EncryptBlock(char const* in, char* result);
//Decrypt exactly one block of ciphertext.
// in - The ciphertext.
// result - The plaintext generated from a ciphertext using the session key.
void DecryptBlock(char const* in, char* result);
void Encrypt(char const* in, char* result, size_t n, int iMode=ECB);
void Decrypt(char const* in, char* result, size_t n, int iMode=ECB);
//Get Key Length
int GetKeyLength()
{
if(false==m_bKeyInit)
throw std::string(sm_szErrorMsg1);
return m_keylength;
}
//Block Size
int GetBlockSize()
{
if(false==m_bKeyInit)
throw std::string(sm_szErrorMsg1);
return m_blockSize;
}
//Number of Rounds
int GetRounds()
{
if(false==m_bKeyInit)
throw std::string(sm_szErrorMsg1);
return m_iROUNDS;
}
void ResetChain()
{
memcpy(m_chain, m_chain0, m_blockSize);
}
public:
//Null chain
static char const* sm_chain0;
private:
static const int sm_alog[256];
static const int sm_log[256];
static const char sm_S[256];
static const char sm_Si[256];
static const int sm_T1[256];
static const int sm_T2[256];
static const int sm_T3[256];
static const int sm_T4[256];
static const int sm_T5[256];
static const int sm_T6[256];
static const int sm_T7[256];
static const int sm_T8[256];
static const int sm_U1[256];
static const int sm_U2[256];
static const int sm_U3[256];
static const int sm_U4[256];
static const char sm_rcon[30];
static const int sm_shifts[3][4][2];
//Error Messages
static char const* sm_szErrorMsg1;
static char const* sm_szErrorMsg2;
//Key Initialization Flag
bool m_bKeyInit;
//Encryption (m_Ke) round key
int m_Ke[MAX_ROUNDS+1][MAX_BC];
//Decryption (m_Kd) round key
int m_Kd[MAX_ROUNDS+1][MAX_BC];
//Key Length
int m_keylength;
//Block Size
int m_blockSize;
//Number of Rounds
int m_iROUNDS;
//Chain Block
char m_chain0[MAX_BLOCK_SIZE];
char m_chain[MAX_BLOCK_SIZE];
//Auxiliary private use buffers
int tk[MAX_KC];
int a[MAX_BC];
int t[MAX_BC];
};
#endif // __RIJNDAEL_H__

1
indra/llimage/llimage.cpp
View File

@@ -1239,6 +1239,7 @@ file_extensions[] =
{
{ "bmp", IMG_CODEC_BMP },
{ "tga", IMG_CODEC_TGA },
{ "j2k", IMG_CODEC_J2C },
{ "j2c", IMG_CODEC_J2C },
{ "jp2", IMG_CODEC_J2C },
{ "texture", IMG_CODEC_J2C },

213
indra/llimage/llimagemetadatareader.cpp
View File

@@ -1,213 +0,0 @@
// <edit>
#include "linden_common.h"
#include "llimagemetadatareader.h"
#include "aes.h"
//#include "llapr.h"
//#include "llerror.h"
const unsigned char EMKDU_AES_KEY[] = {0x01,0x00,0x81,0x07,0x63,0x78,0xB6,0xFE,0x6E,0x3F,0xB0,0x12,0xCC,0x65,0x66,0xC1,
0x81,0x96,0xAC,0xC1,0x3B,0x66,0x0B,0xF7};
//#define COMMENT_DEBUGG1ING
LLJ2cParser::LLJ2cParser(U8* data,int data_size)
{
if(data && data_size)
{
mData.resize(data_size);
memcpy(&(mData[0]), data, data_size);
//std::copy(data,data+data_size,mData.begin());
}
mIter = mData.begin();
}
U8 LLJ2cParser::nextChar()
{
U8 rtn = 0x00;
if(mIter != mData.end())
{
rtn = (*mIter);
mIter++;
}
return rtn;
}
std::vector<U8> LLJ2cParser::nextCharArray(int len)
{
std::vector<U8> array;
if(len > 0)
{
array.resize(len);
for(S32 i = 0; i < len; i++)
{
array[i] = nextChar();
}
}
return array;
}
std::vector<U8> LLJ2cParser::GetNextComment()
{
std::vector<U8> content;
while (mIter != mData.end())
{
U8 marker = nextChar();
if (marker == 0xff)
{
U8 marker_type = nextChar();
if (marker_type == 0x4f)
{
continue;
}
if (marker_type == 0x90)
{
//llinfos << "FOUND 0x90" << llendl;
break; //return empty vector
}
if (marker_type == 0x64)
{
//llinfos << "FOUND 0x64 COMMENT SECTION" << llendl;
S32 len = ((S32)nextChar())*256 + (S32)nextChar();
if (len > 3) content = nextCharArray(len - 2);
return content;
}
}
}
content.clear(); //return empty vector by clear anything there
return content;
}
//flow of control in this method is shit, gotta fix this... possibly return a vector or map instead of a string -HG
/*
Notes:
For anyone debugging this method, if a comment is not being decoded properly and you know encryption is being used,
the easiest thing to do is to create an LLAPRFile handle inside this method and write the contents of data to a file.
Normally the comment is going to be up near the header, just have a look at it in a hex editor.
It's generally going to be a string of 130 bytes preceeded by a null.
*/
//static
unsigned int LLImageMetaDataReader::ExtractEncodedComment(U8* data,int data_size, std::string& output)
{
LLJ2cParser parser = LLJ2cParser(data,data_size);
std::string decodedComment;
//not supported yet, but why the hell not?
unsigned int result = ENC_NONE;
while(1)
{
std::vector<U8> comment = parser.GetNextComment();
if (comment.empty()) break; //exit loop
if (comment[1] == 0x00 && comment.size() == 130)
{
bool xorComment = true;
//llinfos << "FOUND PAYLOAD" << llendl;
std::vector<U8> payload(128);
S32 i;
memcpy(&(payload[0]), &(comment[2]), 128);
//std::copy(comment.begin()+2,comment.end(),payload.begin());
//lets check xorComment Cipher first
if (payload[2] == payload[127])
{
// emkdu.dll
for (i = 4; i < 128; i += 4)
{
payload[i] ^= payload[3];
payload[i + 1] ^= payload[1];
payload[i + 2] ^= payload[0];
payload[i + 3] ^= payload[2];
}
result = ENC_EMKDU_V1;
}
else if (payload[3] == payload[127])
{
// emkdu.dll or onyxkdu.dll
for (i = 4; i < 128; i += 4)
{
payload[i] ^= payload[2];
payload[i + 1] ^= payload[0];
payload[i + 2] ^= payload[1];
payload[i + 3] ^= payload[3];
}
result = ENC_ONYXKDU;
}
else
{
xorComment = false;
}
if(!xorComment)
{
//this is terrible i know
std::vector<U8> decrypted(129);
CRijndael aes;
try
{
aes.MakeKey(reinterpret_cast<const char*>(EMKDU_AES_KEY),"", 24, 16);
} catch(std::string error)
{
llinfos << error << llendl;
}
try
{
int numBlocks = 8;
char* datain = (char*)&(payload[0]);
char* dataout = (char*)&(decrypted[0]);
char buffer[64];
memset(buffer,0,sizeof(buffer));
aes.DecryptBlock(datain,dataout); // do first block
for (int pos = 0; pos < 16; ++pos)
*dataout++ ^= buffer[pos];
datain += 16;
numBlocks--;
while (numBlocks)
{
aes.DecryptBlock(datain,dataout); // do next block
for (int pos = 0; pos < 16; ++pos)
*dataout++ ^= *(datain-16+pos);
datain += 16;
--numBlocks;
}
} catch(std::string error)
{
llinfos << error << llendl;
}
//payload.clear();
//memcpy(&(payload[0]),&(dataout[0]),dataout.size());
for (i = 0 ; i < 128; ++i)
{
if (decrypted[i] == 0) break;
}
if(i == 0) continue;
if(decodedComment.length() > 0)
decodedComment.append(", ");
//the way it's being done now, you can only specify the encryption type for the last comment.
//need to switch to a map<std::string, unsigned int> or a vector for output.
result = ENC_EMKDU_V2;
decodedComment.append(decrypted.begin(),decrypted.begin()+i);
}
else
{
for (i = 4 ; i < 128; ++i)
{
if (payload[i] == 0) break;
}
if(i < 4) continue;
if(decodedComment.length() > 0)
decodedComment.append(", ");
decodedComment.append(payload.begin()+4,payload.begin()+i);
}
//llinfos << "FOUND COMMENT: " << result << llendl;
}
}
//end of loop
output = decodedComment;
return result;
}
// </edit>

32
indra/llimage/llimagemetadatareader.h
View File

@@ -1,32 +0,0 @@
// <edit>
#ifndef LL_LLIMAGEMETADATAREADER_H
#define LL_LLIMAGEMETADATAREADER_H
#include "stdtypes.h"
#include <string.h>
#include <vector>
#include <string>
//encryption types
#define ENC_NONE 0
#define ENC_ONYXKDU 1
#define ENC_EMKDU_V1 2
#define ENC_EMKDU_V2 4
class LLJ2cParser
{
public:
LLJ2cParser(U8* data,int data_size);
std::vector<U8> GetNextComment();
std::vector<U8> mData;
private:
U8 nextChar();
std::vector<U8> nextCharArray(int len);
std::vector<U8>::iterator mIter;
};
class LLImageMetaDataReader
{
public:
static unsigned int ExtractEncodedComment(U8* data,int data_size, std::string& output);
};
#endif
// </edit>

74
indra/llimagej2coj/llimagej2coj.cpp
View File

@@ -415,6 +415,63 @@ BOOL LLImageJ2COJ::encodeImpl(LLImageJ2C &base, const LLImageRaw &raw_image, con
return TRUE;
}
inline S32 extractLong4( U8 const *aBuffer, int nOffset )
{
S32 ret = aBuffer[ nOffset ] << 24;
ret += aBuffer[ nOffset + 1 ] << 16;
ret += aBuffer[ nOffset + 2 ] << 8;
ret += aBuffer[ nOffset + 3 ];
return ret;
}
inline S32 extractShort2( U8 const *aBuffer, int nOffset )
{
S32 ret = aBuffer[ nOffset ] << 8;
ret += aBuffer[ nOffset + 1 ];
return ret;
}
inline bool isSOC( U8 const *aBuffer )
{
return aBuffer[ 0 ] == 0xFF && aBuffer[ 1 ] == 0x4F;
}
inline bool isSIZ( U8 const *aBuffer )
{
return aBuffer[ 0 ] == 0xFF && aBuffer[ 1 ] == 0x51;
}
bool getMetadataFast( LLImageJ2C &aImage, S32 &aW, S32 &aH, S32 &aComps )
{
const int J2K_HDR_LEN( 42 );
const int J2K_HDR_X1( 8 );
const int J2K_HDR_Y1( 12 );
const int J2K_HDR_X0( 16 );
const int J2K_HDR_Y0( 20 );
const int J2K_HDR_NUMCOMPS( 40 );
if( aImage.getDataSize() < J2K_HDR_LEN )
return false;
U8 const* pBuffer = aImage.getData();
if( !isSOC( pBuffer ) || !isSIZ( pBuffer+2 ) )
return false;
S32 x1 = extractLong4( pBuffer, J2K_HDR_X1 );
S32 y1 = extractLong4( pBuffer, J2K_HDR_Y1 );
S32 x0 = extractLong4( pBuffer, J2K_HDR_X0 );
S32 y0 = extractLong4( pBuffer, J2K_HDR_Y0 );
S32 numComps = extractShort2( pBuffer, J2K_HDR_NUMCOMPS );
aComps = numComps;
aW = x1 - x0;
aH = y1 - y0;
return true;
}
BOOL LLImageJ2COJ::getMetadata(LLImageJ2C &base)
{
//
@@ -424,6 +481,18 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2C &base)
// Update the raw discard level
base.updateRawDiscardLevel();
S32 width(0);
S32 height(0);
S32 img_components(0);
if ( getMetadataFast( base, width, height, img_components ) )
{
base.setSize(width, height, img_components);
return TRUE;
}
// Do it the old and slow way, decode the image with openjpeg
opj_dparameters_t parameters; /* decompression parameters */
opj_event_mgr_t event_mgr; /* event manager */
opj_image_t *image = NULL;
@@ -482,12 +551,11 @@ BOOL LLImageJ2COJ::getMetadata(LLImageJ2C &base)
}
// Copy image data into our raw image format (instead of the separate channel format
S32 width = 0;
S32 height = 0;
S32 img_components = image->numcomps;
img_components = image->numcomps;
width = image->x1 - image->x0;
height = image->y1 - image->y0;
base.setSize(width, height, img_components);
/* free image data structure */

7
indra/llmath/CMakeLists.txt
View File

@@ -12,8 +12,7 @@ include_directories(
set(llmath_SOURCE_FILES
llbbox.cpp
llbboxlocal.cpp
llcalc.cpp
llcalcparser.cpp
llcalc.cpp
llcamera.cpp
llcoordframe.cpp
llline.cpp
@@ -48,8 +47,8 @@ set(llmath_HEADER_FILES
coordframe.h
llbbox.h
llbboxlocal.h
llcalc.h
llcalcparser.h
llcalc.h
llcalcparser.h
llcamera.h
llcoord.h
llcoordframe.h

169
indra/llmath/llcalc.cpp
View File

@@ -1,9 +1,26 @@
/*
* LLCalc.cpp
* SecondLife
*
* Created by Aimee Walton on 28/09/2008.
* Copyright 2008 Aimee Walton.
* Copyright 2008 Aimee Walton.
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2008, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*
*/
@@ -11,59 +28,61 @@
#include "llcalc.h"
#include "llcalcparser.h"
#include "llmath.h"
#include "llcalcparser.h"
// Variable names for use in the build floater
const char* LLCalc::X_POS = "PX";
const char* LLCalc::Y_POS = "PY";
const char* LLCalc::Z_POS = "PZ";
const char* LLCalc::X_SCALE = "SX";
const char* LLCalc::Y_SCALE = "SY";
const char* LLCalc::Z_SCALE = "SZ";
const char* LLCalc::X_ROT = "RX";
const char* LLCalc::Y_ROT = "RY";
const char* LLCalc::Z_ROT = "RZ";
const char* LLCalc::HOLLOW = "HLW";
const char* LLCalc::CUT_BEGIN = "CB";
const char* LLCalc::CUT_END = "CE";
const char* LLCalc::PATH_BEGIN = "PB";
const char* LLCalc::PATH_END = "PE";
const char* LLCalc::TWIST_BEGIN = "TB";
const char* LLCalc::TWIST_END = "TE";
const char* LLCalc::X_SHEAR = "SHX";
const char* LLCalc::Y_SHEAR = "SHY";
const char* LLCalc::X_TAPER = "TPX";
const char* LLCalc::Y_TAPER = "TPY";
const char* LLCalc::RADIUS_OFFSET = "ROF";
const char* LLCalc::REVOLUTIONS = "REV";
const char* LLCalc::SKEW = "SKW";
const char* LLCalc::X_HOLE = "HLX";
const char* LLCalc::Y_HOLE = "HLY";
const char* LLCalc::TEX_U_SCALE = "TSU";
const char* LLCalc::TEX_V_SCALE = "TSV";
const char* LLCalc::TEX_U_OFFSET = "TOU";
const char* LLCalc::TEX_V_OFFSET = "TOV";
const char* LLCalc::TEX_ROTATION = "TROT";
const char* LLCalc::TEX_TRANSPARENCY = "TRNS";
const char* LLCalc::TEX_GLOW = "GLOW";
// must be lower case for parser definition
// case-insensitive for actual parsing
const char* LLCalc::X_POS = "px";
const char* LLCalc::Y_POS = "py";
const char* LLCalc::Z_POS = "pz";
const char* LLCalc::X_SCALE = "sx";
const char* LLCalc::Y_SCALE = "sy";
const char* LLCalc::Z_SCALE = "sz";
const char* LLCalc::X_ROT = "rx";
const char* LLCalc::Y_ROT = "ry";
const char* LLCalc::Z_ROT = "rz";
const char* LLCalc::HOLLOW = "hlw";
const char* LLCalc::CUT_BEGIN = "cb";
const char* LLCalc::CUT_END = "ce";
const char* LLCalc::PATH_BEGIN = "pb";
const char* LLCalc::PATH_END = "pe";
const char* LLCalc::TWIST_BEGIN = "tb";
const char* LLCalc::TWIST_END = "te";
const char* LLCalc::X_SHEAR = "shx";
const char* LLCalc::Y_SHEAR = "shy";
const char* LLCalc::X_TAPER = "tpx";
const char* LLCalc::Y_TAPER = "tpy";
const char* LLCalc::RADIUS_OFFSET = "rof";
const char* LLCalc::REVOLUTIONS = "rev";
const char* LLCalc::SKEW = "skw";
const char* LLCalc::X_HOLE = "hlx";
const char* LLCalc::Y_HOLE = "hly";
const char* LLCalc::TEX_U_SCALE = "tsu";
const char* LLCalc::TEX_V_SCALE = "tsv";
const char* LLCalc::TEX_U_OFFSET = "tou";
const char* LLCalc::TEX_V_OFFSET = "tov";
const char* LLCalc::TEX_ROTATION = "trot";
const char* LLCalc::TEX_TRANSPARENCY = "trns";
const char* LLCalc::TEX_GLOW = "glow";
LLCalc* LLCalc::sInstance = NULL;
//TODO: Make this a static global class
LLCalc::LLCalc() : mLastErrorPos(0)
{
// Init table of constants
mConstants["PI"] = F_PI;
mConstants["TWO_PI"] = F_TWO_PI;
mConstants["PI_BY_TWO"] = F_PI_BY_TWO;
mConstants["SQRT_TWO_PI"] = F_SQRT_TWO_PI;
mConstants["SQRT2"] = F_SQRT2;
mConstants["SQRT3"] = F_SQRT3;
mConstants["DEG_TO_RAD"] = DEG_TO_RAD;
mConstants["RAD_TO_DEG"] = RAD_TO_DEG;
mConstants["GRAVITY"] = GRAVITY;
/*setVar("PI", F_PI);
setVar("TWO_PI", F_TWO_PI);
setVar("PI_BY_TWO", F_PI_BY_TWO);
setVar("SQRT_TWO_PI", F_SQRT_TWO_PI);
setVar("SQRT2", F_SQRT2);
setVar("SQRT3", F_SQRT3);
setVar("DEG_TO_RAD", DEG_TO_RAD);
setVar("RAD_TO_DEG", RAD_TO_DEG);
setVar("GRAVITY", GRAVITY);*/
}
LLCalc::~LLCalc()
@@ -80,7 +99,7 @@ void LLCalc::cleanUp()
//static
LLCalc* LLCalc::getInstance()
{
if (!sInstance) sInstance = new LLCalc();
if (!sInstance) sInstance = new LLCalc();
return sInstance;
}
@@ -99,47 +118,35 @@ void LLCalc::clearAllVariables()
mVariables.clear();
}
/*
void LLCalc::updateVariables(LLSD& vars)
{
LLSD::map_iterator cIt = vars.beginMap();
for(; cIt != vars.endMap(); cIt++)
{
setVar(cIt->first, (F32)(LLSD::Real)cIt->second);
}
}
*/
bool LLCalc::evalString(const std::string& expression, F32& result)
{
std::string expr_upper = expression;
LLStringUtil::toUpper(expr_upper);
LLCalcParser calc(result, &mConstants, &mVariables);
mLastErrorPos = 0;
std::string::iterator start = expr_upper.begin();
parse_info<std::string::iterator> info;
try
std::string::const_iterator itr = expression.begin();
expression::grammar<F32,std::string::const_iterator> calc;
calc.constant.add
("pi", F_PI)
("two_pi", F_TWO_PI)
("pi_by_two", F_PI_BY_TWO)
("sqrt_two_pi", F_SQRT_TWO_PI)
("sqrt2", F_SQRT2)
("sqrt3", F_SQRT3)
("deg_to_rad", DEG_TO_RAD)
("rad_to_deg", RAD_TO_DEG)
("gravity", GRAVITY)
;
for(calc_map_t::const_iterator iter = mVariables.begin();
iter != mVariables.end();
++iter)
{
info = parse(start, expr_upper.end(), calc, space_p);
lldebugs << "Math expression: " << expression << " = " << result << llendl;
calc.constant.add(iter->first, iter->second);
}
catch(parser_error<std::string, std::string::iterator> &e)
if (!expression::parse<F32,std::string::const_iterator>(itr, expression.end(), calc, result) || itr != expression.end())
{
mLastErrorPos = e.where - expr_upper.begin();
llinfos << "Calc parser exception: " << e.descriptor << " at " << mLastErrorPos << " in expression: " << expression << llendl;
return false;
}
if (!info.full)
{
mLastErrorPos = info.stop - expr_upper.begin();
mLastErrorPos = itr - expression.begin();
llinfos << "Unhandled syntax error at " << mLastErrorPos << " in expression: " << expression << llendl;
return false;
}
lldebugs << "Math expression: " << expression << " = " << result << llendl;
return true;
}

30
indra/llmath/llcalc.h
View File

@@ -1,9 +1,26 @@
/*
* LLCalc.h
* SecondLife
*
* Created by Aimee Walton on 28/09/2008.
* Copyright 2008 Aimee Walton.
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2008, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*
*/
@@ -69,13 +86,8 @@ public:
private:
std::string::size_type mLastErrorPos;
calc_map_t mConstants;
calc_map_t mVariables;
// *TODO: Add support for storing user defined variables, and stored functions.
// Will need UI work, and a means to save them between sessions.
// calc_map_t* mUserVariables;
// "There shall be only one"
static LLCalc* sInstance;
};

344
indra/llmath/llcalcparser.h
View File

@@ -27,165 +27,207 @@
#ifndef LL_CALCPARSER_H
#define LL_CALCPARSER_H
#include <boost/spirit/include/classic_attribute.hpp>
#include <boost/spirit/include/classic_core.hpp>
#include <boost/spirit/include/classic_error_handling.hpp>
#include <boost/spirit/include/classic_position_iterator.hpp>
#include <boost/spirit/include/phoenix1_binders.hpp>
#include <boost/spirit/include/classic_symbols.hpp>
using namespace boost::spirit::classic;
#include <boost/spirit/version.hpp>
#if !defined(SPIRIT_VERSION) || SPIRIT_VERSION < 0x2010
#error "At least Spirit version 2.1 required"
#endif
#include "llcalc.h"
#include "llmath.h"
// Add this in if we want boost math constants.
//#include <boost/math/constants/constants.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/qi.hpp>
struct LLCalcParser : grammar<LLCalcParser>
namespace expression {
//TODO: If we can find a better way to do this with boost::pheonix::bind lets do it
namespace { // anonymous
template <typename T>
T min_glue(T a, T b)
{
LLCalcParser(F32& result, LLCalc::calc_map_t* constants, LLCalc::calc_map_t* vars) :
mResult(result), mConstants(constants), mVariables(vars) {};
struct value_closure : closure<value_closure, F32>
{
member1 value;
};
template <typename ScannerT>
struct definition
{
// Rule declarations
rule<ScannerT> statement, identifier;
rule<ScannerT, value_closure::context_t> expression, term,
power,
unary_expr,
factor,
unary_func,
binary_func,
group;
return std::min(a, b);
}
// start() should return the starting symbol
rule<ScannerT> const& start() const { return statement; }
definition(LLCalcParser const& self)
template <typename T>
T max_glue(T a, T b)
{
return std::max(a, b);
}
struct lazy_pow_
{
template <typename X, typename Y>
struct result { typedef X type; };
template <typename X, typename Y>
X operator()(X x, Y y) const
{
return std::pow(x, y);
}
};
struct lazy_ufunc_
{
template <typename F, typename A1>
struct result { typedef A1 type; };
template <typename F, typename A1>
A1 operator()(F f, A1 a1) const
{
return f(a1);
}
};
struct lazy_bfunc_
{
template <typename F, typename A1, typename A2>
struct result { typedef A1 type; };
template <typename F, typename A1, typename A2>
A1 operator()(F f, A1 a1, A2 a2) const
{
return f(a1, a2);
}
};
} // end namespace anonymous
template <typename FPT, typename Iterator>
struct grammar
: boost::spirit::qi::grammar<
Iterator, FPT(), boost::spirit::ascii::space_type
>
{
// symbol table for constants
// to be added by the actual calculator
struct constant_
: boost::spirit::qi::symbols<
typename std::iterator_traits<Iterator>::value_type,
FPT
>
{
constant_()
{
using namespace phoenix;
assertion<std::string> assert_domain("Domain error");
// assertion<std::string> assert_symbol("Unknown symbol");
assertion<std::string> assert_syntax("Syntax error");
identifier =
lexeme_d[(alpha_p | '_') >> *(alnum_p | '_')]
;
group =
'(' >> expression[group.value = arg1] >> assert_syntax(ch_p(')'))
;
unary_func =
((str_p("SIN") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_sin)(self,arg1)]) |
(str_p("COS") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_cos)(self,arg1)]) |
(str_p("TAN") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_tan)(self,arg1)]) |
(str_p("ASIN") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_asin)(self,arg1)]) |
(str_p("ACOS") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_acos)(self,arg1)]) |
(str_p("ATAN") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_atan)(self,arg1)]) |
(str_p("SQRT") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_sqrt)(self,arg1)]) |
(str_p("LOG") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_log)(self,arg1)]) |
(str_p("EXP") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_exp)(self,arg1)]) |
(str_p("ABS") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_fabs)(self,arg1)]) |
(str_p("FLR") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_floor)(self,arg1)]) |
(str_p("CEIL") >> '(' >> expression[unary_func.value = bind(&LLCalcParser::_ceil)(self,arg1)])
) >> assert_syntax(ch_p(')'))
;
binary_func =
((str_p("ATAN2") >> '(' >> expression[binary_func.value = arg1] >> ',' >>
expression[binary_func.value = bind(&LLCalcParser::_atan2)(self, binary_func.value, arg1)]) |
(str_p("MIN") >> '(' >> expression[binary_func.value = arg1] >> ',' >>
expression[binary_func.value = bind(&LLCalcParser::_min)(self, binary_func.value, arg1)]) |
(str_p("MAX") >> '(' >> expression[binary_func.value = arg1] >> ',' >>
expression[binary_func.value = bind(&LLCalcParser::_max)(self, binary_func.value, arg1)])
) >> assert_syntax(ch_p(')'))
;
// *TODO: Localisation of the decimal point?
// Problem, LLLineEditor::postvalidateFloat accepts a comma when appropriate
// for the current locale. However to do that here could clash with using
// the comma as a separator when passing arguments to functions.
factor =
(ureal_p[factor.value = arg1] |
group[factor.value = arg1] |
unary_func[factor.value = arg1] |
binary_func[factor.value = arg1] |
// Lookup throws an Unknown Symbol error if it is unknown, while this works fine,
// would be "neater" to handle symbol lookup from here with an assertive parser.
// constants_p[factor.value = arg1]|
identifier[factor.value = bind(&LLCalcParser::lookup)(self, arg1, arg2)]
) >>
// Detect and throw math errors.
assert_domain(eps_p(bind(&LLCalcParser::checkNaN)(self, factor.value)))
;
unary_expr =
!ch_p('+') >> factor[unary_expr.value = arg1] |
'-' >> factor[unary_expr.value = -arg1]
;
power =
unary_expr[power.value = arg1] >>
*('^' >> assert_syntax(unary_expr[power.value = bind(&powf)(power.value, arg1)]))
;
term =
power[term.value = arg1] >>
*(('*' >> assert_syntax(power[term.value *= arg1])) |
('/' >> assert_syntax(power[term.value /= arg1])) |
('%' >> assert_syntax(power[term.value = bind(&fmodf)(term.value, arg1)]))
)
;
expression =
assert_syntax(term[expression.value = arg1]) >>
*(('+' >> assert_syntax(term[expression.value += arg1])) |
('-' >> assert_syntax(term[expression.value -= arg1]))
)
;
statement =
!ch_p('=') >> ( expression )[var(self.mResult) = arg1] >> (end_p)
}
} constant;
// symbol table for unary functions like "abs"
struct ufunc_
: boost::spirit::qi::symbols<
typename std::iterator_traits<Iterator>::value_type,
FPT (*)(FPT)
>
{
ufunc_()
{
this->add
("abs" , (FPT (*)(FPT)) std::abs )
("acos" , (FPT (*)(FPT)) std::acos )
("asin" , (FPT (*)(FPT)) std::asin )
("atan" , (FPT (*)(FPT)) std::atan )
("ceil" , (FPT (*)(FPT)) std::ceil )
("cos" , (FPT (*)(FPT)) std::cos )
("cosh" , (FPT (*)(FPT)) std::cosh )
("exp" , (FPT (*)(FPT)) std::exp )
("floor" , (FPT (*)(FPT)) std::floor)
("log" , (FPT (*)(FPT)) std::log )
("log10" , (FPT (*)(FPT)) std::log10)
("sin" , (FPT (*)(FPT)) std::sin )
("sinh" , (FPT (*)(FPT)) std::sinh )
("sqrt" , (FPT (*)(FPT)) std::sqrt )
("tan" , (FPT (*)(FPT)) std::tan )
("tanh" , (FPT (*)(FPT)) std::tanh )
;
}
};
private:
// Member functions for semantic actions
F32 lookup(const std::string::iterator&, const std::string::iterator&) const;
F32 _min(const F32& a, const F32& b) const { return llmin(a, b); }
F32 _max(const F32& a, const F32& b) const { return llmax(a, b); }
bool checkNaN(const F32& a) const { return !llisnan(a); }
//FIX* non ambigious function fix making SIN() work for calc -Cryogenic Blitz
F32 _sin(const F32& a) const { return sin(DEG_TO_RAD * a); }
F32 _cos(const F32& a) const { return cos(DEG_TO_RAD * a); }
F32 _tan(const F32& a) const { return tan(DEG_TO_RAD * a); }
F32 _asin(const F32& a) const { return asin(a * RAD_TO_DEG); }
F32 _acos(const F32& a) const { return acos(a * RAD_TO_DEG); }
F32 _atan(const F32& a) const { return atan(a * RAD_TO_DEG); }
F32 _sqrt(const F32& a) const { return sqrt(a); }
F32 _log(const F32& a) const { return log(a); }
F32 _exp(const F32& a) const { return exp(a); }
F32 _fabs(const F32& a) const { return fabs(a); }
F32 _floor(const F32& a) const { return (F32)llfloor(a); }
F32 _ceil(const F32& a) const { return llceil(a); }
} ufunc;
F32 _atan2(const F32& a,const F32& b) const { return atan2(a,b); }
LLCalc::calc_map_t* mConstants;
LLCalc::calc_map_t* mVariables;
// LLCalc::calc_map_t* mUserVariables;
F32& mResult;
// symbol table for binary functions like "pow"
struct bfunc_
: boost::spirit::qi::symbols<
typename std::iterator_traits<Iterator>::value_type,
FPT (*)(FPT, FPT)
>
{
bfunc_()
{
using boost::bind;
this->add
("pow" , (FPT (*)(FPT, FPT)) std::pow )
("atan2", (FPT (*)(FPT, FPT)) std::atan2)
("min" , (FPT (*)(FPT, FPT)) min_glue)
("max" , (FPT (*)(FPT, FPT)) max_glue)
;
}
} bfunc;
boost::spirit::qi::rule<
Iterator, FPT(), boost::spirit::ascii::space_type
> expression, term, factor, primary;
grammar() : grammar::base_type(expression)
{
using boost::spirit::qi::real_parser;
using boost::spirit::qi::real_policies;
real_parser<FPT,real_policies<FPT> > real;
using boost::spirit::qi::_1;
using boost::spirit::qi::_2;
using boost::spirit::qi::_3;
using boost::spirit::qi::no_case;
using boost::spirit::qi::_val;
boost::phoenix::function<lazy_pow_> lazy_pow;
boost::phoenix::function<lazy_ufunc_> lazy_ufunc;
boost::phoenix::function<lazy_bfunc_> lazy_bfunc;
expression =
term [_val = _1]
>> *( ('+' >> term [_val += _1])
| ('-' >> term [_val -= _1])
)
;
term =
factor [_val = _1]
>> *( ('*' >> factor [_val *= _1])
| ('/' >> factor [_val /= _1])
)
;
factor =
primary [_val = _1]
>> *( ("**" >> factor [_val = lazy_pow(_val, _1)])
)
;
primary =
real [_val = _1]
| '(' >> expression [_val = _1] >> ')'
| ('-' >> primary [_val = -_1])
| ('+' >> primary [_val = _1])
| (no_case[ufunc] >> '(' >> expression >> ')')
[_val = lazy_ufunc(_1, _2)]
| (no_case[bfunc] >> '(' >> expression >> ','
>> expression >> ')')
[_val = lazy_bfunc(_1, _2, _3)]
| no_case[constant] [_val = _1]
;
}
};
template <typename FPT, typename Iterator>
bool parse(Iterator &iter,
Iterator end,
const grammar<FPT,Iterator> &g,
FPT &result)
{
return boost::spirit::qi::phrase_parse(
iter, end, g, boost::spirit::ascii::space, result);
}
} // end namespace expression
#endif // LL_CALCPARSER_H

14
indra/llmath/lloctree.h
View File

@@ -41,6 +41,7 @@
#if LL_RELEASE_WITH_DEBUG_INFO || LL_DEBUG
#define OCT_ERRS LL_ERRS("OctreeErrors")
#define OCTREE_GUARD_CHECK
#else
#define OCT_ERRS LL_WARNS("OctreeErrors")
#endif
@@ -183,6 +184,7 @@ public:
virtual void traverse(const LLOctreeNode<T>* node);
};
#ifdef OCTREE_GUARD_CHECK
struct OctreeGuard
{
template <typename T>
@@ -210,6 +212,18 @@ struct OctreeGuard
}
void* mNode;
};
#else
struct OctreeGuard
{
template <typename T>
OctreeGuard(const LLOctreeNode<T>* node) {}
~OctreeGuard() {}
template <typename T>
static bool checkGuarded(const LLOctreeNode<T>* node) {return false;}
};
#endif
template <class T>
class LLOctreeNode : public LLTreeNode<T>
{

46
indra/llmessage/CMakeLists.txt
View File

@@ -4,20 +4,32 @@ project(llmessage)
include(00-Common)
include(LLCommon)
include(AIStateMachine)
include(LLMath)
include(LLMessage)
include(LLVFS)
include(LLXML)
include_directories (${CMAKE_CURRENT_SOURCE_DIR})
include_directories(
${LLCOMMON_INCLUDE_DIRS}
${AISTATEMACHINE_INCLUDE_DIRS}
${LLMATH_INCLUDE_DIRS}
${LLMESSAGE_INCLUDE_DIRS}
${LLVFS_INCLUDE_DIRS}
${LLXML_INCLUDE_DIRS}
)
set(llmessage_SOURCE_FILES
aicurl.cpp
aicurleasyrequeststatemachine.cpp
aicurlperhost.cpp
aicurlthread.cpp
aihttpheaders.cpp
aihttptimeoutpolicy.cpp
debug_libcurl.cpp
llhttpclient.cpp
llares.cpp
llareslistener.cpp
llassetstorage.cpp
@@ -29,15 +41,10 @@ set(llmessage_SOURCE_FILES
llchainio.cpp
llcircuit.cpp
llclassifiedflags.cpp
aicurl.cpp
debug_libcurl.cpp
aicurlthread.cpp
lldatapacker.cpp
lldispatcher.cpp
llfiltersd2xmlrpc.cpp
llhost.cpp
llhttpclient.cpp
llhttpclientadapter.cpp
llhttpnode.cpp
llhttpsender.cpp
llinstantmessage.cpp
@@ -49,6 +56,7 @@ set(llmessage_SOURCE_FILES
llmail.cpp
llmessagebuilder.cpp
llmessageconfig.cpp
llmessagelog.cpp
llmessagereader.cpp
llmessagetemplate.cpp
llmessagetemplateparser.cpp
@@ -62,14 +70,11 @@ set(llmessage_SOURCE_FILES
llpartdata.cpp
llproxy.cpp
llpumpio.cpp
llregionpresenceverifier.cpp
llsdappservices.cpp
llsdhttpserver.cpp
llsdmessage.cpp
llsdmessagebuilder.cpp
llsdmessagereader.cpp
llservicebuilder.cpp
llservice.cpp
llstoredmessage.cpp
lltemplatemessagebuilder.cpp
lltemplatemessagedispatcher.cpp
@@ -94,7 +99,6 @@ set(llmessage_SOURCE_FILES
message_prehash.cpp
message_string_table.cpp
net.cpp
partsyspacket.cpp
patch_code.cpp
patch_dct.cpp
patch_idct.cpp
@@ -104,6 +108,14 @@ set(llmessage_SOURCE_FILES
set(llmessage_HEADER_FILES
CMakeLists.txt
aicurl.h
aicurleasyrequeststatemachine.h
aicurlprivate.h
aicurlperhost.h
aicurlthread.h
aihttpheaders.h
aihttptimeoutpolicy.h
debug_libcurl.h
llares.h
llareslistener.h
llassetstorage.h
@@ -117,10 +129,6 @@ set(llmessage_HEADER_FILES
llcircuit.h
llclassifiedflags.h
llcurl.h
aicurl.h
debug_libcurl.h
aicurlprivate.h
aicurlthread.h
lldatapacker.h
lldbstrings.h
lldispatcher.h
@@ -129,8 +137,6 @@ set(llmessage_HEADER_FILES
llfollowcamparams.h
llhost.h
llhttpclient.h
llhttpclientinterface.h
llhttpclientadapter.h
llhttpnode.h
llhttpnodeadapter.h
llhttpsender.h
@@ -145,6 +151,7 @@ set(llmessage_HEADER_FILES
llmail.h
llmessagebuilder.h
llmessageconfig.h
llmessagelog.h
llmessagereader.h
llmessagetemplate.h
llmessagetemplateparser.h
@@ -162,14 +169,11 @@ set(llmessage_HEADER_FILES
llqueryflags.h
llregionflags.h
llregionhandle.h
llregionpresenceverifier.h
llsdappservices.h
llsdhttpserver.h
llsdmessage.h
llsdmessagebuilder.h
llsdmessagereader.h
llservice.h
llservicebuilder.h
llstoredmessage.h
lltaskname.h
llteleportflags.h
@@ -197,7 +201,6 @@ set(llmessage_HEADER_FILES
message.h
message_prehash.h
net.h
partsyspacket.h
patch_code.h
patch_dct.h
sound_ids.h
@@ -227,12 +230,10 @@ if (LL_TESTS)
include(Tut)
SET(llmessage_TEST_SOURCE_FILES
# llhttpclientadapter.cpp
llmime.cpp
llnamevalue.cpp
lltrustedmessageservice.cpp
lltemplatemessagedispatcher.cpp
llregionpresenceverifier.cpp
)
LL_ADD_PROJECT_UNIT_TESTS(llmessage "${llmessage_TEST_SOURCE_FILES}")
@@ -243,7 +244,8 @@ if (LL_TESTS)
${LLVFS_LIBRARIES}
${LLMATH_LIBRARIES}
${LLCOMMON_LIBRARIES}
${GOOGLEMOCK_LIBRARIES}
${GOOGLEMOCK_LIBRARIES}
${LLXML_LIBRARIES}
)
LL_ADD_INTEGRATION_TEST(

771
indra/llmessage/aicurl.cpp
View File

File diff suppressed because it is too large Load Diff

269
indra/llmessage/aicurl.h
View File

@@ -36,6 +36,7 @@
#include <set>
#include <stdexcept>
#include <boost/intrusive_ptr.hpp>
#include <boost/shared_ptr.hpp>
#include <boost/utility.hpp>
#include "llpreprocessor.h"
@@ -48,12 +49,51 @@
#undef CURLOPT_DNS_USE_GLOBAL_CACHE
#define CURLOPT_DNS_USE_GLOBAL_CACHE do_not_use_CURLOPT_DNS_USE_GLOBAL_CACHE
#include "stdtypes.h" // U32
#include "lliopipe.h" // LLIOPipe::buffer_ptr_t
#include "stdtypes.h" // U16, S32, U32, F64
#include "llatomic.h" // LLAtomicU32
#include "aithreadsafe.h"
#include "llhttpstatuscodes.h"
#include "llhttpclient.h"
// Debug Settings.
extern bool gNoVerifySSLCert;
class LLSD;
class LLBufferArray;
class LLChannelDescriptors;
class AIHTTPTimeoutPolicy;
// Some pretty printing for curl easy handle related things:
// Print the lock object related to the current easy handle in every debug output.
#ifdef CWDEBUG
#include <libcwd/buf2str.h>
#include <sstream>
#define DoutCurl(x) do { \
using namespace libcwd; \
std::ostringstream marker; \
marker << (void*)this->get_lockobj(); \
libcw_do.push_marker(); \
libcw_do.marker().assign(marker.str().data(), marker.str().size()); \
libcw_do.inc_indent(2); \
Dout(dc::curl, x); \
libcw_do.dec_indent(2); \
libcw_do.pop_marker(); \
} while(0)
#define DoutCurlEntering(x) do { \
using namespace libcwd; \
std::ostringstream marker; \
marker << (void*)this->get_lockobj(); \
libcw_do.push_marker(); \
libcw_do.marker().assign(marker.str().data(), marker.str().size()); \
libcw_do.inc_indent(2); \
DoutEntering(dc::curl, x); \
libcw_do.dec_indent(2); \
libcw_do.pop_marker(); \
} while(0)
#else // !CWDEBUG
#define DoutCurl(x) Dout(dc::curl, x << " [" << (void*)this->get_lockobj() << ']')
#define DoutCurlEntering(x) DoutEntering(dc::curl, x << " [" << (void*)this->get_lockobj() << ']')
#endif // CWDEBUG
//-----------------------------------------------------------------------------
// Exceptions.
@@ -76,40 +116,59 @@ class AICurlNoMultiHandle : public AICurlError {
AICurlNoMultiHandle(std::string const& message) : AICurlError(message) { }
};
class AICurlNoBody : public AICurlError {
public:
AICurlNoBody(std::string const& message) : AICurlError(message) { }
};
// End Exceptions.
//-----------------------------------------------------------------------------
// Things defined in this namespace are called from elsewhere in the viewer code.
namespace AICurlInterface {
// Output parameter of AICurlPrivate::CurlEasyRequest::getResult.
// Only used by LLXMLRPCTransaction::Impl.
struct TransferInfo {
TransferInfo() : mSizeDownload(0.0), mTotalTime(0.0), mSpeedDownload(0.0) { }
F64 mSizeDownload;
F64 mTotalTime;
F64 mSpeedDownload;
struct Stats {
static LLAtomicU32 easy_calls;
static LLAtomicU32 easy_errors;
static LLAtomicU32 easy_init_calls;
static LLAtomicU32 easy_init_errors;
static LLAtomicU32 easy_cleanup_calls;
static LLAtomicU32 multi_calls;
static LLAtomicU32 multi_errors;
static LLAtomicU32 AICurlEasyRequest_count;
static LLAtomicU32 AICurlEasyRequestStateMachine_count;
static LLAtomicU32 BufferedCurlEasyRequest_count;
static LLAtomicU32 ResponderBase_count;
static LLAtomicU32 ThreadSafeBufferedCurlEasyRequest_count;
static LLAtomicU32 status_count[100];
static LLAtomicU32 llsd_body_count;
static LLAtomicU32 llsd_body_parse_error;
static LLAtomicU32 raw_body_count;
static void print(void);
static U32 status2index(U32 status);
static U32 index2status(U32 index);
};
//-----------------------------------------------------------------------------
// Global functions.
// Called to handle changes in Debug Settings.
bool handleCurlMaxTotalConcurrentConnections(LLSD const& newvalue);
bool handleCurlConcurrentConnectionsPerHost(LLSD const& newvalue);
bool handleNoVerifySSLCert(LLSD const& newvalue);
// Called once at start of application (from newview/llappviewer.cpp by main thread (before threads are created)),
// with main purpose to initialize curl.
void initCurl(void (*)(void) = NULL);
void initCurl(void);
// Called once at start of application (from LLAppViewer::initThreads), starts AICurlThread.
void startCurlThread(void);
void startCurlThread(U32 CurlMaxTotalConcurrentConnections, U32 CurlConcurrentConnectionsPerHost, bool NoVerifySSLCert);
// Called once at end of application (from newview/llappviewer.cpp by main thread),
// with purpose to stop curl threads, free curl resources and deinitialize curl.
void cleanupCurl(void);
// Called from indra/llmessage/llurlrequest.cpp to print debug output regarding
// an error code returned by EasyRequest::getResult.
// Just returns curl_easy_strerror(errorcode).
std::string strerror(CURLcode errorcode);
// Called from indra/newview/llfloaterabout.cpp for the About floater, and
// from newview/llappviewer.cpp in behalf of debug output.
// Just returns curl_version().
@@ -123,105 +182,11 @@ void setCAFile(std::string const& file);
// Can be used to set the path to the Certificate Authority file.
void setCAPath(std::string const& file);
//-----------------------------------------------------------------------------
// Global classes.
// Responder - base class for Request::get* and Request::post API.
//
// The life cycle of classes derived from this class is as follows:
// They are allocated with new on the line where get(), getByteRange() or post() is called,
// and the pointer to the allocated object is then put in a reference counting ResponderPtr.
// This ResponderPtr is passed to CurlResponderBuffer::prepRequest which stores it in its
// member mResponder. Hence, the life time of a Responder is never longer than its
// associated CurlResponderBuffer, however, if everything works correctly, then normally a
// responder is deleted in CurlResponderBuffer::removed_from_multi_handle by setting
// mReponder to NULL.
//
// Note that the lifetime of CurlResponderBuffer is (a bit) shorter than the associated
// CurlEasyRequest (because of the order of base classes of ThreadSafeBufferedCurlEasyRequest)
// and the callbacks, as set by prepRequest, only use those two.
// A callback locks the CurlEasyRequest before actually making the callback, and the
// destruction of CurlResponderBuffer also first locks the CurlEasyRequest, and then revokes
// the callbacks. This assures that a Responder is never used when the objects it uses are
// destructed. Also, if any of those are destructed then the Responder is automatically
// destructed too.
//
class Responder {
protected:
Responder(void);
virtual ~Responder();
private:
// Associated URL, used for debug output.
std::string mURL;
public:
// Called to set the URL of the current request for this Responder,
// used only when printing debug output regarding activity of the Responder.
void setURL(std::string const& url);
public:
// Called from LLHTTPClientURLAdaptor::complete():
// Derived classes can override this to get the HTML header that was received, when the message is completed.
// The default does nothing.
virtual void completedHeader(U32 status, std::string const& reason, LLSD const& content);
// Derived classes can override this to get the raw data of the body of the HTML message that was received.
// The default is to interpret the content as LLSD and call completed().
virtual void completedRaw(U32 status, std::string const& reason, LLChannelDescriptors const& channels, LLIOPipe::buffer_ptr_t const& buffer);
// Called from LLHTTPClient request calls, if an error occurs even before we can call one of the above.
// It calls completed() with a fake status U32_MAX, as that is what some derived clients expect (bad design).
// This means that if a derived class overrides completedRaw() it now STILL has to override completed() to catch this error.
void fatalError(std::string const& reason);
// A derived class should return true if curl should follow redirections.
// The default is not to follow redirections.
virtual bool followRedir(void) { return false; }
protected:
// ... or, derived classes can override this to get the LLSD content when the message is completed.
// The default is to call result() (or errorWithContent() in case of a HTML status indicating an error).
virtual void completed(U32 status, std::string const& reason, LLSD const& content);
// ... or, derived classes can override this to received the content of a body upon success.
// The default does nothing.
virtual void result(LLSD const& content);
// Derived classes can override this to get informed when a bad HTML status code is received.
// The default calls error().
virtual void errorWithContent(U32 status, std::string const& reason, LLSD const& content);
// ... or, derived classes can override this to get informed when a bad HTML statis code is received.
// The default prints the error to llinfos.
virtual void error(U32 status, std::string const& reason);
public:
// Called from LLSDMessage::ResponderAdapter::listener.
// LLSDMessage::ResponderAdapter is a hack, showing among others by fact that these functions need to be public.
void pubErrorWithContent(U32 status, std::string const& reason, LLSD const& content) { errorWithContent(status, reason, content); }
void pubResult(LLSD const& content) { result(content); }
private:
// Used by ResponderPtr. Object is deleted when reference count reaches zero.
LLAtomicU32 mReferenceCount;
friend void intrusive_ptr_add_ref(Responder* p); // Called by boost::intrusive_ptr when a new copy of a boost::intrusive_ptr<Responder> is made.
friend void intrusive_ptr_release(Responder* p); // Called by boost::intrusive_ptr when a boost::intrusive_ptr<Responder> is destroyed.
// This function must delete the Responder object when the reference count reaches zero.
};
// A Responder is passed around as ResponderPtr, which causes it to automatically
// destruct when there are no pointers left pointing to it.
typedef boost::intrusive_ptr<Responder> ResponderPtr;
} // namespace AICurlInterface
// Forward declaration (see aicurlprivate.h).
namespace AICurlPrivate {
class CurlEasyRequest;
class BufferedCurlEasyRequest;
} // namespace AICurlPrivate
// Define access types (_crat = Const Read Access Type, _rat = Read Access Type, _wat = Write Access Type).
@@ -229,13 +194,13 @@ namespace AICurlPrivate {
// AICurlEasyRequest h1; // Create easy handle.
// AICurlEasyRequest h2(h1); // Make lightweight copies.
// AICurlEasyRequest_wat h2_w(*h2); // Lock and obtain write access to the easy handle.
// Use *h2_w, which is a reference to the locked CurlEasyRequest instance.
// Use *h2_w, which is a reference to the locked BufferedCurlEasyRequest instance.
// Note: As it is not allowed to use curl easy handles in any way concurrently,
// read access would at most give access to a CURL const*, which will turn out
// to be completely useless; therefore it is sufficient and efficient to use
// an AIThreadSafeSimple and it's unlikely that AICurlEasyRequest_rat will be used.
typedef AIAccessConst<AICurlPrivate::CurlEasyRequest> AICurlEasyRequest_rat;
typedef AIAccess<AICurlPrivate::CurlEasyRequest> AICurlEasyRequest_wat;
typedef AIAccessConst<AICurlPrivate::BufferedCurlEasyRequest> AICurlEasyRequest_rat;
typedef AIAccess<AICurlPrivate::BufferedCurlEasyRequest> AICurlEasyRequest_wat;
// Events generated by AICurlPrivate::CurlEasyHandle.
struct AICurlEasyHandleEvents {
@@ -264,32 +229,41 @@ typedef LLPointer<AIPostField> AIPostFieldPtr;
#include "aicurlprivate.h"
// AICurlPrivate::CurlEasyRequestPtr, a boost::intrusive_ptr, is no more threadsafe than a
// AICurlPrivate::BufferedCurlEasyRequestPtr, a boost::intrusive_ptr, is no more threadsafe than a
// builtin type, but wrapping it in AIThreadSafe is obviously not going to help here.
// Therefore we use the following trick: we wrap CurlEasyRequestPtr too, and only allow
// Therefore we use the following trick: we wrap BufferedCurlEasyRequestPtr too, and only allow
// read accesses on it.
// AICurlEasyRequest: a thread safe, reference counting, auto-cleaning curl easy handle.
// AICurlEasyRequest: a thread safe, reference counting, buffered, auto-cleaning curl easy handle.
class AICurlEasyRequest {
public:
private:
// Use AICurlEasyRequestStateMachine, not AICurlEasyRequest.
friend class AICurlEasyRequestStateMachine;
// Initial construction is allowed (thread-safe).
// Note: If ThreadSafeCurlEasyRequest() throws then the memory allocated is still freed.
// 'new' never returned however and neither the constructor nor destructor of mCurlEasyRequest is called in this case.
// Note: If ThreadSafeBufferedCurlEasyRequest() throws then the memory allocated is still freed.
// 'new' never returned however and neither the constructor nor destructor of mBufferedCurlEasyRequest is called in this case.
// This might throw AICurlNoEasyHandle.
AICurlEasyRequest(bool buffered) :
mCurlEasyRequest(buffered ? new AICurlPrivate::ThreadSafeBufferedCurlEasyRequest : new AICurlPrivate::ThreadSafeCurlEasyRequest) { }
AICurlEasyRequest(AICurlEasyRequest const& orig) : mCurlEasyRequest(orig.mCurlEasyRequest) { }
AICurlEasyRequest(void) :
mBufferedCurlEasyRequest(new AICurlPrivate::ThreadSafeBufferedCurlEasyRequest) { AICurlInterface::Stats::AICurlEasyRequest_count++; }
public:
// Update stats.
~AICurlEasyRequest() { --AICurlInterface::Stats::AICurlEasyRequest_count; }
// Used for storing this object in a standard container (see MultiHandle::add_easy_request).
AICurlEasyRequest(AICurlEasyRequest const& orig) : mBufferedCurlEasyRequest(orig.mBufferedCurlEasyRequest) { AICurlInterface::Stats::AICurlEasyRequest_count++; }
// For the rest, only allow read operations.
AIThreadSafeSimple<AICurlPrivate::CurlEasyRequest>& operator*(void) const { llassert(mCurlEasyRequest.get()); return *mCurlEasyRequest; }
AIThreadSafeSimple<AICurlPrivate::CurlEasyRequest>* operator->(void) const { llassert(mCurlEasyRequest.get()); return mCurlEasyRequest.get(); }
AIThreadSafeSimple<AICurlPrivate::CurlEasyRequest>* get(void) const { return mCurlEasyRequest.get(); }
AIThreadSafeSimple<AICurlPrivate::BufferedCurlEasyRequest>& operator*(void) const { llassert(mBufferedCurlEasyRequest.get()); return *mBufferedCurlEasyRequest; }
AIThreadSafeSimple<AICurlPrivate::BufferedCurlEasyRequest>* operator->(void) const { llassert(mBufferedCurlEasyRequest.get()); return mBufferedCurlEasyRequest.get(); }
AIThreadSafeSimple<AICurlPrivate::BufferedCurlEasyRequest>* get(void) const { return mBufferedCurlEasyRequest.get(); }
// Returns true if this object points to the same CurlEasyRequest object.
bool operator==(AICurlEasyRequest const& cer) const { return mCurlEasyRequest == cer.mCurlEasyRequest; }
// Returns true if this object points to the same BufferedCurlEasyRequest object.
bool operator==(AICurlEasyRequest const& cer) const { return mBufferedCurlEasyRequest == cer.mBufferedCurlEasyRequest; }
// Returns true if this object points to a different CurlEasyRequest object.
bool operator!=(AICurlEasyRequest const& cer) const { return mCurlEasyRequest != cer.mCurlEasyRequest; }
// Returns true if this object points to a different BufferedCurlEasyRequest object.
bool operator!=(AICurlEasyRequest const& cer) const { return mBufferedCurlEasyRequest != cer.mBufferedCurlEasyRequest; }
// Queue this request for insertion in the multi session.
void addRequest(void);
@@ -297,12 +271,9 @@ class AICurlEasyRequest {
// Queue a command to remove this request from the multi session (or cancel a queued command to add it).
void removeRequest(void);
// Returns true when this AICurlEasyRequest wraps a AICurlPrivate::ThreadSafeBufferedCurlEasyRequest.
bool isBuffered(void) const { return mCurlEasyRequest->isBuffered(); }
private:
// The actual pointer to the ThreadSafeCurlEasyRequest instance.
AICurlPrivate::CurlEasyRequestPtr mCurlEasyRequest;
// The actual pointer to the ThreadSafeBufferedCurlEasyRequest instance.
AICurlPrivate::BufferedCurlEasyRequestPtr mBufferedCurlEasyRequest;
private:
// Assignment would not be thread-safe; we may create this object and read from it.
@@ -310,32 +281,28 @@ class AICurlEasyRequest {
// destruct it while another thread still needs it, concurrent or not.
AICurlEasyRequest& operator=(AICurlEasyRequest const&) { return *this; }
public:
// Instead of assignment, it might be helpful to use swap.
void swap(AICurlEasyRequest& cer) { mBufferedCurlEasyRequest.swap(cer.mBufferedCurlEasyRequest); }
public:
// The more exotic member functions of this class, to deal with passing this class
// as CURLOPT_PRIVATE pointer to a curl handle and afterwards restore it.
// For "internal use" only; don't use things from AICurlPrivate yourself.
// It's thread-safe to give read access the underlaying boost::intrusive_ptr.
// It's not OK to then call get() on that and store the AICurlPrivate::ThreadSafeCurlEasyRequest* separately.
AICurlPrivate::CurlEasyRequestPtr const& get_ptr(void) const { return mCurlEasyRequest; }
// It's not OK to then call get() on that and store the AICurlPrivate::ThreadSafeBufferedCurlEasyRequest* separately.
AICurlPrivate::BufferedCurlEasyRequestPtr const& get_ptr(void) const { return mBufferedCurlEasyRequest; }
// If we have a correct (with regard to reference counting) AICurlPrivate::CurlEasyRequestPtr,
// If we have a correct (with regard to reference counting) AICurlPrivate::BufferedCurlEasyRequestPtr,
// then it's OK to construct a AICurlEasyRequest from it.
// Note that the external AICurlPrivate::CurlEasyRequestPtr needs its own locking, because
// Note that the external AICurlPrivate::BufferedCurlEasyRequestPtr needs its own locking, because
// it's not thread-safe in itself.
AICurlEasyRequest(AICurlPrivate::CurlEasyRequestPtr const& ptr) : mCurlEasyRequest(ptr) { }
AICurlEasyRequest(AICurlPrivate::BufferedCurlEasyRequestPtr const& ptr) : mBufferedCurlEasyRequest(ptr) { AICurlInterface::Stats::AICurlEasyRequest_count++; }
// This one is obviously dangerous. It's for use only in MultiHandle::check_run_count.
// See also the long comment in CurlEasyRequest::finalizeRequest with regard to CURLOPT_PRIVATE.
explicit AICurlEasyRequest(AICurlPrivate::ThreadSafeCurlEasyRequest* ptr) : mCurlEasyRequest(ptr) { }
};
// Write Access Type for the buffer.
struct AICurlResponderBuffer_wat : public AIAccess<AICurlPrivate::CurlResponderBuffer> {
explicit AICurlResponderBuffer_wat(AICurlPrivate::ThreadSafeBufferedCurlEasyRequest& lockobj) :
AIAccess<AICurlPrivate::CurlResponderBuffer>(lockobj) { }
AICurlResponderBuffer_wat(AIThreadSafeSimple<AICurlPrivate::CurlEasyRequest>& lockobj) :
AIAccess<AICurlPrivate::CurlResponderBuffer>(static_cast<AICurlPrivate::ThreadSafeBufferedCurlEasyRequest&>(lockobj)) { }
// This one is obviously dangerous. It's for use only in MultiHandle::check_msg_queue.
// See also the long comment in BufferedCurlEasyRequest::finalizeRequest with regard to CURLOPT_PRIVATE.
explicit AICurlEasyRequest(AICurlPrivate::ThreadSafeBufferedCurlEasyRequest* ptr) : mBufferedCurlEasyRequest(ptr) { AICurlInterface::Stats::AICurlEasyRequest_count++; }
};
#define AICurlPrivate DONTUSE_AICurlPrivate

72
indra/newview/statemachine/aicurleasyrequeststatemachine.cpp → indra/llmessage/aicurleasyrequeststatemachine.cpp
View File

@@ -30,6 +30,7 @@
#include "linden_common.h"
#include "aicurleasyrequeststatemachine.h"
#include "aihttptimeoutpolicy.h"
#include "llcontrol.h"
enum curleasyrequeststatemachine_state_type {
@@ -61,8 +62,8 @@ void AICurlEasyRequestStateMachine::initialize_impl(void)
{
{
AICurlEasyRequest_wat curlEasyRequest_w(*mCurlEasyRequest);
llassert(curlEasyRequest_w->is_finalized()); // Call finalizeRequest(url) before calling run().
curlEasyRequest_w->send_events_to(this);
llassert(curlEasyRequest_w->is_finalized()); // Call finalizeRequest() before calling run().
curlEasyRequest_w->send_handle_events_to(this);
}
mAdded = false;
mTimedOut = false;
@@ -87,6 +88,9 @@ void AICurlEasyRequestStateMachine::finished(AICurlEasyRequest_wat&)
// CURL-THREAD
void AICurlEasyRequestStateMachine::removed_from_multi_handle(AICurlEasyRequest_wat&)
{
llassert(mFinished || mTimedOut); // If we neither finished nor timed out, then why is this being removed?
// Note that allowing this would cause an assertion later on for removing
// a BufferedCurlEasyRequest with a still active Responder.
set_state(mFinished ? AICurlEasyRequestStateMachine_removed_after_finished : AICurlEasyRequestStateMachine_removed);
}
@@ -116,13 +120,15 @@ void AICurlEasyRequestStateMachine::multiplex_impl(void)
// 3) AICurlEasyRequestStateMachine_finished (running)
// 4) AICurlEasyRequestStateMachine_removed_after_finished (running)
// Set an inactivity timer.
// This shouldn't really be necessary, except in the case of a bug
// in libcurl; but lets be sure and set a timer for inactivity.
static LLCachedControl<F32> CurlRequestTimeOut("CurlRequestTimeOut", 40.f);
mTimer = new AIPersistentTimer; // Do not delete timer upon expiration.
mTimer->setInterval(CurlRequestTimeOut);
mTimer->run(this, AICurlEasyRequestStateMachine_timedOut, false, false);
if (mTotalDelayTimeout > 0.f)
{
// Set an inactivity timer.
// This shouldn't really be necessary, except in the case of a bug
// in libcurl; but lets be sure and set a timer for inactivity.
mTimer = new AIPersistentTimer; // Do not delete timer upon expiration.
mTimer->setInterval(mTotalDelayTimeout);
mTimer->run(this, AICurlEasyRequestStateMachine_timedOut, false, false);
}
break;
}
case AICurlEasyRequestStateMachine_added:
@@ -161,17 +167,16 @@ void AICurlEasyRequestStateMachine::multiplex_impl(void)
// Only do this once.
mHandled = true;
// Stop the timer. Note that it's the main thread that generates timer events,
// so we're certain that there will be no time out anymore if we reach this point.
mTimer->abort();
if (mTimer)
{
// Stop the timer. Note that it's the main thread that generates timer events,
// so we're certain that there will be no time out anymore if we reach this point.
mTimer->abort();
}
// The request finished and either data or an error code is available.
if (mBuffered)
{
AICurlEasyRequest_wat easy_request_w(*mCurlEasyRequest);
AICurlResponderBuffer_wat buffered_easy_request_w(*mCurlEasyRequest);
buffered_easy_request_w->processOutput(easy_request_w);
}
AICurlEasyRequest_wat easy_request_w(*mCurlEasyRequest);
easy_request_w->processOutput();
}
if (current_state == AICurlEasyRequestStateMachine_finished)
@@ -187,11 +192,10 @@ void AICurlEasyRequestStateMachine::multiplex_impl(void)
case AICurlEasyRequestStateMachine_removed:
{
// The request was removed from the multi handle.
if (mBuffered && mTimedOut)
if (mTimedOut)
{
AICurlEasyRequest_wat easy_request_w(*mCurlEasyRequest);
AICurlResponderBuffer_wat buffered_easy_request_w(*mCurlEasyRequest);
buffered_easy_request_w->timed_out();
easy_request_w->timed_out();
}
// We're done. If we timed out, abort -- or else the application will
@@ -226,24 +230,36 @@ void AICurlEasyRequestStateMachine::finish_impl(void)
// Revoke callbacks.
{
AICurlEasyRequest_wat curl_easy_request_w(*mCurlEasyRequest);
curl_easy_request_w->send_events_to(NULL);
curl_easy_request_w->send_buffer_events_to(NULL);
curl_easy_request_w->send_handle_events_to(NULL);
curl_easy_request_w->revokeCallbacks();
}
// Note that even if the timer expired, it wasn't deleted because we used AIPersistentTimer; so mTimer is still valid.
// Stop the timer, if it's still running.
if (!mHandled)
mTimer->abort();
if (mTimer)
{
// Note that even if the timer expired, it wasn't deleted because we used AIPersistentTimer; so mTimer is still valid.
// Stop the timer, if it's still running.
if (!mHandled)
mTimer->abort();
}
// Auto clean up ourselves.
kill();
}
AICurlEasyRequestStateMachine::AICurlEasyRequestStateMachine(bool buffered) : mBuffered(buffered), mCurlEasyRequest(buffered)
AICurlEasyRequestStateMachine::AICurlEasyRequestStateMachine(void) :
mTimer(NULL), mTotalDelayTimeout(AIHTTPTimeoutPolicy::getDebugSettingsCurlTimeout().getTotalDelay())
{
Dout(dc::statemachine, "Calling AICurlEasyRequestStateMachine(" << (buffered ? "true" : "false") << ") [" << (void*)this << "] [" << (void*)mCurlEasyRequest.get() << "]");
Dout(dc::statemachine, "Calling AICurlEasyRequestStateMachine(void) [" << (void*)this << "] [" << (void*)mCurlEasyRequest.get() << "]");
AICurlInterface::Stats::AICurlEasyRequestStateMachine_count++;
}
void AICurlEasyRequestStateMachine::setTotalDelayTimeout(F32 totalDelayTimeout)
{
mTotalDelayTimeout = totalDelayTimeout;
}
AICurlEasyRequestStateMachine::~AICurlEasyRequestStateMachine()
{
Dout(dc::statemachine, "Calling ~AICurlEasyRequestStateMachine() [" << (void*)this << "] [" << (void*)mCurlEasyRequest.get() << "]");
--AICurlInterface::Stats::AICurlEasyRequestStateMachine_count;
}

20
indra/newview/statemachine/aicurleasyrequeststatemachine.h → indra/llmessage/aicurleasyrequeststatemachine.h
View File

@@ -40,7 +40,7 @@
// Before calling cersm.run() initialize the object (cersm) as follows:
//
// AICurlEasyRequest_wat cersm_w(cersm);
// cersm_w->setopt(...); // etc, see the interface of AICurlPrivate::CurlEasyRequest and it's base class AICurlPrivate::CurlEasyHandle.
// cersm_w->setopt(...); // etc, see the interface of AICurlPrivate::CurlEasyRequest.
//
// When the state machine finishes, call aborted() to check
// whether or not the statemachine succeeded in fetching
@@ -52,18 +52,22 @@
// Construction of a AICurlEasyRequestStateMachine might throw AICurlNoEasyHandle.
class AICurlEasyRequestStateMachine : public AIStateMachine, public AICurlEasyHandleEvents {
public:
AICurlEasyRequestStateMachine(bool buffered);
AICurlEasyRequestStateMachine(void);
// Transparent access.
AICurlEasyRequest mCurlEasyRequest;
private:
bool mBuffered; // Argument used for construction of mCurlEasyRequest.
bool mAdded; // Set when the last command to the curl thread was to add the request.
bool mTimedOut; // Set if the expiration timer timed out.
bool mFinished; // Set by the curl thread to signal it finished.
bool mHandled; // Set when we processed the received data.
AITimer* mTimer; // Expiration timer.
bool mAdded; // Set when the last command to the curl thread was to add the request.
bool mTimedOut; // Set if the expiration timer timed out.
bool mFinished; // Set by the curl thread to signal it finished.
bool mHandled; // Set when we processed the received data.
AITimer* mTimer; // Expiration timer.
F32 mTotalDelayTimeout; // The time out value for mTimer.
public:
// Called to set a specific time out, instead of the default one.
void setTotalDelayTimeout(F32 totalDelayTimeout);
protected:
// AICurlEasyRequest Events.

175
indra/llmessage/aicurlperhost.cpp Normal file
View File

@@ -0,0 +1,175 @@
/**
* @file aiperhost.cpp
* @brief Implementation of PerHostRequestQueue
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 04/11/2012
* Initial version, written by Aleric Inglewood @ SL
*/
#include "sys.h"
#include "aicurlperhost.h"
#include "aicurlthread.h"
#undef AICurlPrivate
namespace AICurlPrivate {
PerHostRequestQueue::threadsafe_instance_map_type PerHostRequestQueue::sInstanceMap;
U32 curl_concurrent_connections_per_host;
//static
PerHostRequestQueuePtr PerHostRequestQueue::instance(std::string const& hostname)
{
llassert(!hostname.empty());
instance_map_wat instance_map_w(sInstanceMap);
PerHostRequestQueue::iterator iter = instance_map_w->find(hostname);
if (iter == instance_map_w->end())
{
iter = instance_map_w->insert(instance_map_type::value_type(hostname, new RefCountedThreadSafePerHostRequestQueue)).first;
}
// Note: the creation of PerHostRequestQueuePtr MUST be protected by the lock on sInstanceMap (see release()).
return iter->second;
}
//static
void PerHostRequestQueue::release(PerHostRequestQueuePtr& instance)
{
if (instance->exactly_two_left()) // Being 'instance' and the one in sInstanceMap.
{
// The viewer can be have left main() we can't access the global sInstanceMap anymore.
if (LLApp::isStopped())
{
return;
}
instance_map_wat instance_map_w(sInstanceMap);
// It is possible that 'exactly_two_left' is not up to date anymore.
// Therefore, recheck the condition now that we have locked sInstanceMap.
if (!instance->exactly_two_left())
{
// Some other thread added this host in the meantime.
return;
}
// The reference in the map is the last one; that means there can't be any curl easy requests queued for this host.
llassert(PerHostRequestQueue_wat(*instance)->mQueuedRequests.empty());
// Find the host and erase it from the map.
iterator const end = instance_map_w->end();
for(iterator iter = instance_map_w->begin(); iter != end; ++iter)
{
if (instance == iter->second)
{
instance_map_w->erase(iter);
instance.reset();
return;
}
}
// We should always find the host.
llassert(false);
}
instance.reset();
}
bool PerHostRequestQueue::throttled() const
{
llassert(mAdded <= curl_concurrent_connections_per_host);
return mAdded == curl_concurrent_connections_per_host;
}
void PerHostRequestQueue::added_to_multi_handle(void)
{
llassert(mAdded < curl_concurrent_connections_per_host);
++mAdded;
}
void PerHostRequestQueue::removed_from_multi_handle(void)
{
--mAdded;
llassert(mAdded >= 0);
}
void PerHostRequestQueue::queue(AICurlEasyRequest const& easy_request)
{
mQueuedRequests.push_back(easy_request.get_ptr());
}
bool PerHostRequestQueue::cancel(AICurlEasyRequest const& easy_request)
{
queued_request_type::iterator const end = mQueuedRequests.end();
queued_request_type::iterator cur = std::find(mQueuedRequests.begin(), end, easy_request.get_ptr());
if (cur == end)
return false; // Not found.
// We can't use erase because that uses assignment to move elements,
// because it isn't thread-safe. Therefore, move the element that we found to
// the back with swap (could just swap with the end immediately, but I don't
// want to break the order in which requests where added). Swap is also not
// thread-safe, but OK here because it only touches the objects in the deque,
// and the deque is protected by the lock on the PerHostRequestQueue object.
queued_request_type::iterator prev = cur;
while (++cur != end)
{
prev->swap(*cur); // This is safe,
prev = cur;
}
mQueuedRequests.pop_back(); // if this is safe.
return true;
}
void PerHostRequestQueue::add_queued_to(curlthread::MultiHandle* multi_handle)
{
if (!mQueuedRequests.empty())
{
multi_handle->add_easy_request(mQueuedRequests.front());
mQueuedRequests.pop_front();
}
}
//static
void PerHostRequestQueue::purge(void)
{
instance_map_wat instance_map_w(sInstanceMap);
for (iterator host = instance_map_w->begin(); host != instance_map_w->end(); ++host)
{
Dout(dc::curl, "Purging queue of host \"" << host->first << "\".");
PerHostRequestQueue_wat(*host->second)->mQueuedRequests.clear();
}
}
// Friend functions of RefCountedThreadSafePerHostRequestQueue
void intrusive_ptr_add_ref(RefCountedThreadSafePerHostRequestQueue* per_host)
{
per_host->mReferenceCount++;
}
void intrusive_ptr_release(RefCountedThreadSafePerHostRequestQueue* per_host)
{
if (--per_host->mReferenceCount == 0)
{
delete per_host;
}
}
} // namespace AICurlPrivate

135
indra/llmessage/aicurlperhost.h Normal file
View File

@@ -0,0 +1,135 @@
/**
* @file aicurlperhost.h
* @brief Definition of class PerHostRequestQueue
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 04/11/2012
* Initial version, written by Aleric Inglewood @ SL
*/
#ifndef AICURLPERHOST_H
#define AICURLPERHOST_H
#include "llerror.h" // llassert
#include <string>
#include <deque>
#include <map>
#include <boost/intrusive_ptr.hpp>
#include "aithreadsafe.h"
class AICurlEasyRequest;
namespace AICurlPrivate {
namespace curlthread { class MultiHandle; }
class PerHostRequestQueue;
class RefCountedThreadSafePerHostRequestQueue;
class ThreadSafeBufferedCurlEasyRequest;
// Forward declaration of BufferedCurlEasyRequestPtr (see aicurlprivate.h).
typedef boost::intrusive_ptr<ThreadSafeBufferedCurlEasyRequest> BufferedCurlEasyRequestPtr;
// PerHostRequestQueue objects are created by the curl thread and destructed by the main thread.
// We need locking.
typedef AIThreadSafeSimpleDC<PerHostRequestQueue> threadsafe_PerHostRequestQueue;
typedef AIAccessConst<PerHostRequestQueue> PerHostRequestQueue_crat;
typedef AIAccess<PerHostRequestQueue> PerHostRequestQueue_rat;
typedef AIAccess<PerHostRequestQueue> PerHostRequestQueue_wat;
// We can't put threadsafe_PerHostRequestQueue in a std::map because you can't copy a mutex.
// Therefore, use an intrusive pointer for the threadsafe type.
typedef boost::intrusive_ptr<RefCountedThreadSafePerHostRequestQueue> PerHostRequestQueuePtr;
//-----------------------------------------------------------------------------
// PerHostRequestQueue
// This class provides a static interface to create and maintain instances
// of PerHostRequestQueue objects, so that at any moment there is at most
// one instance per hostname. Those instances then are used to queue curl
// requests when the maximum number of connections for that host already
// have been reached.
class PerHostRequestQueue {
private:
typedef std::map<std::string, PerHostRequestQueuePtr> instance_map_type;
typedef AIThreadSafeSimpleDC<instance_map_type> threadsafe_instance_map_type;
typedef AIAccess<instance_map_type> instance_map_rat;
typedef AIAccess<instance_map_type> instance_map_wat;
static threadsafe_instance_map_type sInstanceMap; // Map of PerHostRequestQueue instances with the hostname as key.
friend class AIThreadSafeSimpleDC<PerHostRequestQueue>; //threadsafe_PerHostRequestQueue
PerHostRequestQueue(void) : mAdded(0) { }
public:
typedef instance_map_type::iterator iterator;
typedef instance_map_type::const_iterator const_iterator;
// Return (possibly create) a unique instance for the given hostname.
static PerHostRequestQueuePtr instance(std::string const& hostname);
// Release instance (object will be deleted if this was the last instance).
static void release(PerHostRequestQueuePtr& instance);
// Remove everything. Called upon viewer exit.
static void purge(void);
private:
typedef std::deque<BufferedCurlEasyRequestPtr> queued_request_type;
int mAdded; // Number of active easy handles with this host.
queued_request_type mQueuedRequests; // Waiting (throttled) requests.
public:
void added_to_multi_handle(void); // Called when an easy handle for this host has been added to the multi handle.
void removed_from_multi_handle(void); // Called when an easy handle for this host is removed again from the multi handle.
bool throttled(void) const; // Returns true if the maximum number of allowed requests for this host have been added to the multi handle.
void queue(AICurlEasyRequest const& easy_request); // Add easy_request to the queue.
bool cancel(AICurlEasyRequest const& easy_request); // Remove easy_request from the queue (if it's there).
void add_queued_to(curlthread::MultiHandle* mh); // Add queued easy handle (if any) to the multi handle. The request is removed from the queue,
// followed by either a call to added_to_multi_handle() or to queue() to add it back.
private:
// Disallow copying.
PerHostRequestQueue(PerHostRequestQueue const&) { }
};
class RefCountedThreadSafePerHostRequestQueue : public threadsafe_PerHostRequestQueue {
public:
RefCountedThreadSafePerHostRequestQueue(void) : mReferenceCount(0) { }
bool exactly_two_left(void) const { return mReferenceCount == 2; }
private:
// Used by PerHostRequestQueuePtr. Object is deleted when reference count reaches zero.
LLAtomicU32 mReferenceCount;
friend void intrusive_ptr_add_ref(RefCountedThreadSafePerHostRequestQueue* p);
friend void intrusive_ptr_release(RefCountedThreadSafePerHostRequestQueue* p);
};
extern U32 curl_concurrent_connections_per_host;
} // namespace AICurlPrivate
#endif // AICURLPERHOST_H

350
indra/llmessage/aicurlprivate.h
View File

@@ -33,34 +33,96 @@
#include <sstream>
#include "llatomic.h"
#include "llrefcount.h"
#include "aicurlperhost.h"
class AIHTTPHeaders;
class AIHTTPTimeoutPolicy;
class AICurlEasyRequest;
class AICurlEasyRequestStateMachine;
namespace AICurlPrivate {
namespace curlthread { class MultiHandle; }
struct Stats {
static LLAtomicU32 easy_calls;
static LLAtomicU32 easy_errors;
static LLAtomicU32 easy_init_calls;
static LLAtomicU32 easy_init_errors;
static LLAtomicU32 easy_cleanup_calls;
static LLAtomicU32 multi_calls;
static LLAtomicU32 multi_errors;
class CurlEasyRequest;
class ThreadSafeBufferedCurlEasyRequest;
static void print(void);
namespace curlthread {
class MultiHandle;
// A class that keeps track of timeout administration per connection.
class HTTPTimeout : public LLRefCount {
private:
AIHTTPTimeoutPolicy const* mPolicy; // A pointer to the used timeout policy.
std::vector<U32> mBuckets; // An array with the number of bytes transfered in each second.
U16 mBucket; // The bucket corresponding to mLastSecond.
bool mNothingReceivedYet; // Set when created, reset when the HTML reply header from the server is received.
bool mLowSpeedOn; // Set while uploading or downloading data.
bool mUploadFinished; // Used to keep track of whether upload_finished was called yet.
S32 mLastSecond; // The time at which lowspeed() was last called, in seconds since mLowSpeedClock.
U32 mTotalBytes; // The sum of all bytes in mBuckets.
U64 mLowSpeedClock; // Clock count at which low speed detection (re)started.
U64 mStalled; // The clock count at which this transaction is considered to be stalling if nothing is transfered anymore.
public:
static F64 const sClockWidth; // Time between two clock ticks in seconds.
static U64 sClockCount; // Clock count used as 'now' during one loop of the main loop.
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
ThreadSafeBufferedCurlEasyRequest* mLockObj;
#endif
public:
HTTPTimeout(AIHTTPTimeoutPolicy const* policy, ThreadSafeBufferedCurlEasyRequest* lock_obj) :
mPolicy(policy), mNothingReceivedYet(true), mLowSpeedOn(false), mUploadFinished(false), mStalled((U64)-1)
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
, mLockObj(lock_obj)
#endif
{ }
// Called after sending all headers, when body data is written the first time.
void connected(void);
// Called when everything we had to send to the server has been sent.
void upload_finished(void);
// Called when data is sent. Returns true if transfer timed out.
bool data_sent(size_t n);
// Called when data is received. Returns true if transfer timed out.
bool data_received(size_t n);
// Called immediately before done() after curl finished, with code.
void done(AICurlEasyRequest_wat const& curlEasyRequest_w, CURLcode code);
// Accessor.
bool has_stalled(void) const { return mStalled < sClockCount; }
// Called from BufferedCurlEasyRequest::processOutput if a timeout occurred.
void print_diagnostics(CurlEasyRequest const* curl_easy_request, char const* eff_url);
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
void* get_lockobj(void) const { return mLockObj; }
#endif
private:
// (Re)start low speed transer rate detection.
void reset_lowspeed(void);
// Common low speed detection, Called from data_sent or data_received.
bool lowspeed(size_t bytes);
};
} // namespace curlthread
void handle_multi_error(CURLMcode code);
inline CURLMcode check_multi_code(CURLMcode code) { Stats::multi_calls++; if (code != CURLM_OK) handle_multi_error(code); return code; }
inline CURLMcode check_multi_code(CURLMcode code) { AICurlInterface::Stats::multi_calls++; if (code != CURLM_OK) handle_multi_error(code); return code; }
bool curlThreadIsRunning(void);
void wakeUpCurlThread(void);
void stopCurlThread(void);
class ThreadSafeCurlEasyRequest;
class ThreadSafeBufferedCurlEasyRequest;
void clearCommandQueue(void);
#define DECLARE_SETOPT(param_type) \
CURLcode setopt(CURLoption option, param_type parameter)
CURLcode setopt(CURLoption option, param_type parameter)
// This class wraps CURL*'s.
// It guarantees that a pointer is cleaned up when no longer needed, as required by libcurl.
@@ -112,20 +174,20 @@ class CurlEasyHandle : public boost::noncopyable, protected AICurlEasyHandleEven
// Extract information from a curl handle.
private:
CURLcode getinfo_priv(CURLINFO info, void* data);
CURLcode getinfo_priv(CURLINFO info, void* data) const;
public:
// The rest are inlines to provide some type-safety.
CURLcode getinfo(CURLINFO info, char** data) { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, curl_slist** data) { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, double* data) { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, long* data) { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, char** data) const { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, curl_slist** data) const { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, double* data) const { return getinfo_priv(info, data); }
CURLcode getinfo(CURLINFO info, long* data) const { return getinfo_priv(info, data); }
#ifdef __LP64__ // sizeof(long) > sizeof(int) ?
// Overload for integer types that are too small (libcurl demands a long).
CURLcode getinfo(CURLINFO info, S32* data) { long ldata; CURLcode res = getinfo_priv(info, &ldata); *data = static_cast<S32>(ldata); return res; }
CURLcode getinfo(CURLINFO info, U32* data) { long ldata; CURLcode res = getinfo_priv(info, &ldata); *data = static_cast<U32>(ldata); return res; }
CURLcode getinfo(CURLINFO info, S32* data) const { long ldata; CURLcode res = getinfo_priv(info, &ldata); *data = static_cast<S32>(ldata); return res; }
CURLcode getinfo(CURLINFO info, U32* data) const { long ldata; CURLcode res = getinfo_priv(info, &ldata); *data = static_cast<U32>(ldata); return res; }
#else // sizeof(long) == sizeof(int)
CURLcode getinfo(CURLINFO info, S32* data) { return getinfo_priv(info, reinterpret_cast<long*>(data)); }
CURLcode getinfo(CURLINFO info, U32* data) { return getinfo_priv(info, reinterpret_cast<long*>(data)); }
CURLcode getinfo(CURLINFO info, S32* data) const { return getinfo_priv(info, reinterpret_cast<long*>(data)); }
CURLcode getinfo(CURLINFO info, U32* data) const { return getinfo_priv(info, reinterpret_cast<long*>(data)); }
#endif
// Perform a file transfer (blocking).
@@ -142,7 +204,7 @@ class CurlEasyHandle : public boost::noncopyable, protected AICurlEasyHandleEven
private:
CURL* mEasyHandle;
CURLM* mActiveMultiHandle;
char* mErrorBuffer;
mutable char* mErrorBuffer;
AIPostFieldPtr mPostField; // This keeps the POSTFIELD data alive for as long as the easy handle exists.
bool mQueuedForRemoval; // Set if the easy handle is (probably) added to the multi handle, but is queued for removal.
#ifdef SHOW_ASSERT
@@ -160,10 +222,6 @@ class CurlEasyHandle : public boost::noncopyable, protected AICurlEasyHandleEven
// Returns true if this easy handle was added to a curl multi handle.
bool active(void) const { return mActiveMultiHandle; }
// If there was an error code as result, then this returns a human readable error string.
// Only valid when setErrorBuffer was called and the curl_easy function returned an error.
std::string getErrorString(void) const { return mErrorBuffer ? mErrorBuffer : "(null)"; }
// Returns true when it is expected that the parent will revoke callbacks before the curl
// easy handle is removed from the multi handle; that usually happens when an external
// error demands termination of the request (ie, an expiration).
@@ -174,14 +232,14 @@ class CurlEasyHandle : public boost::noncopyable, protected AICurlEasyHandleEven
private:
// Call this prior to every curl_easy function whose return value is passed to check_easy_code.
void setErrorBuffer(void);
void setErrorBuffer(void) const;
static void handle_easy_error(CURLcode code);
// Always first call setErrorBuffer()!
static inline CURLcode check_easy_code(CURLcode code)
{
Stats::easy_calls++;
AICurlInterface::Stats::easy_calls++;
if (code != CURLE_OK)
handle_easy_error(code);
return code;
@@ -203,23 +261,23 @@ class CurlEasyHandle : public boost::noncopyable, protected AICurlEasyHandleEven
// to set the options on a curl easy handle.
//
// Calling sendRequest() will then connect to the given URL and perform
// the data exchange. If an error occurs related to this handle, it can
// be read by calling getErrorString().
// the data exchange. Use getResult() to determine if an error occurred.
//
// Note that the life cycle of a CurlEasyRequest is controlled by AICurlEasyRequest:
// a CurlEasyRequest is only ever created as base class of a ThreadSafeCurlEasyRequest,
// a CurlEasyRequest is only ever created as base class of a ThreadSafeBufferedCurlEasyRequest,
// which is only created by creating a AICurlEasyRequest. When the last copy of such
// AICurlEasyRequest is deleted, then also the ThreadSafeCurlEasyRequest is deleted
// AICurlEasyRequest is deleted, then also the ThreadSafeBufferedCurlEasyRequest is deleted
// and the CurlEasyRequest destructed.
class CurlEasyRequest : public CurlEasyHandle {
private:
void setPost_raw(S32 size, char const* data);
void setPost_raw(U32 size, char const* data);
public:
void setPost(S32 size) { setPost_raw(size, NULL); }
void setPost(AIPostFieldPtr const& postdata, S32 size);
void setPost(char const* data, S32 size) { setPost(new AIPostField(data), size); }
void setPost(U32 size) { setPost_raw(size, NULL); }
void setPost(AIPostFieldPtr const& postdata, U32 size);
void setPost(char const* data, U32 size) { setPost(new AIPostField(data), size); }
void setoptString(CURLoption option, std::string const& value);
void addHeader(char const* str);
void addHeaders(AIHTTPHeaders const& headers);
private:
// Callback stubs.
@@ -246,62 +304,110 @@ class CurlEasyRequest : public CurlEasyHandle {
// Call this if the set callbacks are about to be invalidated.
void revokeCallbacks(void);
protected:
// Reset everything to the state it was in when this object was just created.
// Called by BufferedCurlEasyRequest::resetState.
void resetState(void);
private:
// Called from applyDefaultOptions.
void applyProxySettings(void);
// Used in applyProxySettings.
// Used in applyDefaultOptions.
static CURLcode curlCtxCallback(CURL* curl, void* sslctx, void* parm);
// Called from get_timeout_object and httptimeout.
void create_timeout_object(void);
public:
// Set default options that we want applied to all curl easy handles.
void applyDefaultOptions(void);
// Prepare the request for adding it to a multi session, or calling perform.
// This actually adds the headers that were collected with addHeader.
void finalizeRequest(std::string const& url);
void finalizeRequest(std::string const& url, AIHTTPTimeoutPolicy const& policy, AICurlEasyRequestStateMachine* state_machine);
// Store result code that is returned by getResult.
void store_result(CURLcode result) { mResult = result; }
// Last second initialization. Called from MultiHandle::add_easy_request.
void set_timeout_opts(void);
// Called when the curl easy handle is done.
void done(AICurlEasyRequest_wat& curl_easy_request_w) { finished(curl_easy_request_w); }
public:
// Called by MultiHandle::finish_easy_request() to store result code that is returned by getResult.
void storeResult(CURLcode result) { mResult = result; }
// Fill info with the transfer info.
void getTransferInfo(AICurlInterface::TransferInfo* info);
// Called by MultiHandle::finish_easy_request() when the curl easy handle is done.
void done(AICurlEasyRequest_wat& curl_easy_request_w, CURLcode result)
{
if (mTimeout)
{
// Update timeout administration.
mTimeout->done(curl_easy_request_w, result);
}
finished(curl_easy_request_w);
}
// Called by MultiHandle::check_msg_queue() to fill info with the transfer info.
void getTransferInfo(AITransferInfo* info);
// If result != CURLE_FAILED_INIT then also info was filled.
void getResult(CURLcode* result, AICurlInterface::TransferInfo* info = NULL);
void getResult(CURLcode* result, AITransferInfo* info = NULL);
private:
// For debugging purposes.
void print_curl_timings(void) const;
protected:
curl_slist* mHeaders;
bool mRequestFinalized;
AICurlEasyHandleEvents* mEventsTarget;
CURLcode mResult;
AICurlEasyHandleEvents* mHandleEventsTarget;
CURLcode mResult; //AIFIXME: this does not belong in the request object, but belongs in the response object.
private:
// This class may only be created by constructing a ThreadSafeCurlEasyRequest.
friend class ThreadSafeCurlEasyRequest;
AIHTTPTimeoutPolicy const* mTimeoutPolicy;
std::string mLowercaseHostname; // Lowercase hostname (canonicalized) extracted from the url.
PerHostRequestQueuePtr mPerHostPtr; // Pointer to the corresponding PerHostRequestQueue.
LLPointer<curlthread::HTTPTimeout> mTimeout;// Timeout administration object associated with last created CurlSocketInfo.
bool mTimeoutIsOrphan; // Set to true when mTimeout is not (yet) associated with a CurlSocketInfo.
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
public:
bool mDebugIsHeadOrGetMethod;
#endif
public:
// These two are only valid after finalizeRequest.
AIHTTPTimeoutPolicy const* getTimeoutPolicy(void) const { return mTimeoutPolicy; }
std::string const& getLowercaseHostname(void) const { return mLowercaseHostname; }
// Called by CurlSocketInfo to allow access to the last (after a redirect) HTTPTimeout object related to this request.
// This creates mTimeout (unless mTimeoutIsOrphan is set in which case it adopts the orphan).
LLPointer<curlthread::HTTPTimeout>& get_timeout_object(void);
// Accessor for mTimeout with optional creation of orphaned object (if lockobj != NULL).
LLPointer<curlthread::HTTPTimeout>& httptimeout(void) { if (!mTimeout) { create_timeout_object(); mTimeoutIsOrphan = true; } return mTimeout; }
// Return true if no data has been received on the latest socket (if any) for too long.
bool has_stalled(void) const { return mTimeout && mTimeout->has_stalled(); }
protected:
// This class may only be created as base class of BufferedCurlEasyRequest.
// Throws AICurlNoEasyHandle.
CurlEasyRequest(void) :
mHeaders(NULL), mRequestFinalized(false), mEventsTarget(NULL), mResult(CURLE_FAILED_INIT)
{ applyDefaultOptions(); }
CurlEasyRequest(void) : mHeaders(NULL), mHandleEventsTarget(NULL), mResult(CURLE_FAILED_INIT), mTimeoutPolicy(NULL), mTimeoutIsOrphan(false)
#if defined(CWDEBUG) || defined(DEBUG_CURLIO)
, mDebugIsHeadOrGetMethod(false)
#endif
{ applyDefaultOptions(); }
public:
~CurlEasyRequest();
public:
// Post-initialization, set the parent to pass the events to.
void send_events_to(AICurlEasyHandleEvents* target) { mEventsTarget = target; }
void send_handle_events_to(AICurlEasyHandleEvents* target) { mHandleEventsTarget = target; }
// For debugging purposes
bool is_finalized(void) const { return mRequestFinalized; }
bool is_finalized(void) const { return mTimeoutPolicy; }
// Return pointer to the ThreadSafe (wrapped) version of this object.
ThreadSafeCurlEasyRequest* get_lockobj(void);
inline ThreadSafeBufferedCurlEasyRequest* get_lockobj(void);
inline ThreadSafeBufferedCurlEasyRequest const* get_lockobj(void) const;
// PerHost API.
PerHostRequestQueuePtr getPerHostPtr(void); // (Optionally create and) return a pointer to the unique
// PerHostRequestQueue corresponding to mLowercaseHostname.
bool removeFromPerHostQueue(AICurlEasyRequest const&) const; // Remove this request from the per-host queue, if queued at all.
// Returns true if it was queued.
protected:
// Pass events to parent.
/*virtual*/ void added_to_multi_handle(AICurlEasyRequest_wat& curl_easy_request_w);
@@ -309,47 +415,48 @@ class CurlEasyRequest : public CurlEasyHandle {
/*virtual*/ void removed_from_multi_handle(AICurlEasyRequest_wat& curl_easy_request_w);
};
// Buffers used by the AICurlInterface::Request API.
// Curl callbacks write into and read from these buffers.
// The interface with the rest of the code is through AICurlInterface::Responder.
//
// The lifetime of a CurlResponderBuffer is slightly shorter than its
// associated CurlEasyRequest; this class can only be created as base class
// of ThreadSafeBufferedCurlEasyRequest, and is therefore constructed after
// the construction of the associated CurlEasyRequest and destructed before it.
// Hence, it's safe to use get_lockobj() and through that access the CurlEasyRequest
// object at all times.
//
// A CurlResponderBuffer is thus created when a ThreadSafeBufferedCurlEasyRequest
// is created which only happens by creating a AICurlEasyRequest(true) instance,
// and when the last AICurlEasyRequest is deleted, then the ThreadSafeBufferedCurlEasyRequest
// is deleted and the CurlResponderBuffer destructed.
class CurlResponderBuffer : protected AICurlEasyHandleEvents {
// This class adds input/output buffers to the request and hooks up the libcurl callbacks to use those buffers.
// Received data is partially decoded and made available through various member functions.
class BufferedCurlEasyRequest : public CurlEasyRequest {
public:
void resetState(AICurlEasyRequest_wat& curl_easy_request_w);
void prepRequest(AICurlEasyRequest_wat& buffered_curl_easy_request_w, std::vector<std::string> const& headers, AICurlInterface::ResponderPtr responder, S32 time_out = 0, bool post = false);
// The type of the used buffers.
typedef boost::shared_ptr<LLBufferArray> buffer_ptr_t;
LLIOPipe::buffer_ptr_t& getInput(void) { return mInput; }
std::stringstream& getHeaderOutput(void) { return mHeaderOutput; }
LLIOPipe::buffer_ptr_t& getOutput(void) { return mOutput; }
void resetState(void);
void prepRequest(AICurlEasyRequest_wat& buffered_curl_easy_request_w, AIHTTPHeaders const& headers, LLHTTPClient::ResponderPtr responder);
// Called if libcurl doesn't deliver within CurlRequestTimeOut seconds.
buffer_ptr_t& getInput(void) { return mInput; }
buffer_ptr_t& getOutput(void) { return mOutput; }
// Called if libcurl doesn't deliver within AIHTTPTimeoutPolicy::mMaximumTotalDelay seconds.
void timed_out(void);
// Called after removed_from_multi_handle was called.
void processOutput(AICurlEasyRequest_wat& curl_easy_request_w);
void processOutput(void);
// Do not write more than this amount.
//void setBodyLimit(U32 size) { mBodyLimit = size; }
// Post-initialization, set the parent to pass the events to.
void send_buffer_events_to(AIBufferedCurlEasyRequestEvents* target) { mBufferEventsTarget = target; }
protected:
/*virtual*/ void added_to_multi_handle(AICurlEasyRequest_wat& curl_easy_request_w);
/*virtual*/ void finished(AICurlEasyRequest_wat& curl_easy_request_w);
/*virtual*/ void removed_from_multi_handle(AICurlEasyRequest_wat& curl_easy_request_w);
// Events from this class.
/*virtual*/ void received_HTTP_header(void);
/*virtual*/ void received_header(std::string const& key, std::string const& value);
/*virtual*/ void completed_headers(U32 status, std::string const& reason, AITransferInfo* info);
private:
LLIOPipe::buffer_ptr_t mInput;
buffer_ptr_t mInput;
U8* mLastRead; // Pointer into mInput where we last stopped reading (or NULL to start at the beginning).
std::stringstream mHeaderOutput;
LLIOPipe::buffer_ptr_t mOutput;
AICurlInterface::ResponderPtr mResponder;
buffer_ptr_t mOutput;
LLHTTPClient::ResponderPtr mResponder;
//U32 mBodyLimit; // From the old LLURLRequestDetail::mBodyLimit, but never used.
U32 mStatus; // HTTP status, decoded from the first header line.
std::string mReason; // The "reason" from the same header line.
S32 mRequestTransferedBytes;
S32 mResponseTransferedBytes;
AIBufferedCurlEasyRequestEvents* mBufferEventsTarget;
public:
static LLChannelDescriptors const sChannels; // Channel object for mInput (channel out()) and mOutput (channel in()).
@@ -357,64 +464,65 @@ class CurlResponderBuffer : protected AICurlEasyHandleEvents {
private:
// This class may only be created by constructing a ThreadSafeBufferedCurlEasyRequest.
friend class ThreadSafeBufferedCurlEasyRequest;
CurlResponderBuffer(void);
BufferedCurlEasyRequest(void);
public:
~CurlResponderBuffer();
~BufferedCurlEasyRequest();
private:
static size_t curlWriteCallback(char* data, size_t size, size_t nmemb, void* user_data);
static size_t curlReadCallback(char* data, size_t size, size_t nmemb, void* user_data);
static size_t curlHeaderCallback(char* data, size_t size, size_t nmemb, void* user_data);
// Called from curlHeaderCallback.
void setStatusAndReason(U32 status, std::string const& reason);
// Called from processOutput by in case of an error.
void print_diagnostics(CURLcode code);
public:
// Return pointer to the ThreadSafe (wrapped) version of this object.
ThreadSafeBufferedCurlEasyRequest* get_lockobj(void);
ThreadSafeBufferedCurlEasyRequest const* get_lockobj(void) const;
// Return true when prepRequest was already called and the object has not been
// invalidated as a result of calling timed_out().
bool isValid(void) const { return mResponder; }
};
// This class wraps CurlEasyRequest for thread-safety and adds a reference counter so we can
inline ThreadSafeBufferedCurlEasyRequest* CurlEasyRequest::get_lockobj(void)
{
return static_cast<BufferedCurlEasyRequest*>(this)->get_lockobj();
}
inline ThreadSafeBufferedCurlEasyRequest const* CurlEasyRequest::get_lockobj(void) const
{
return static_cast<BufferedCurlEasyRequest const*>(this)->get_lockobj();
}
// This class wraps BufferedCurlEasyRequest for thread-safety and adds a reference counter so we can
// copy it around cheaply and it gets destructed automatically when the last instance is deleted.
// It guarantees that the CURL* handle is never used concurrently, which is not allowed by libcurl.
// As AIThreadSafeSimpleDC contains a mutex, it cannot be copied. Therefore we need a reference counter for this object.
class ThreadSafeCurlEasyRequest : public AIThreadSafeSimple<CurlEasyRequest> {
class ThreadSafeBufferedCurlEasyRequest : public AIThreadSafeSimple<BufferedCurlEasyRequest> {
public:
// Throws AICurlNoEasyHandle.
ThreadSafeCurlEasyRequest(void) : mReferenceCount(0)
{ new (ptr()) CurlEasyRequest;
Dout(dc::curl, "Creating ThreadSafeCurlEasyRequest with this = " << (void*)this); }
virtual ~ThreadSafeCurlEasyRequest()
{ Dout(dc::curl, "Destructing ThreadSafeCurlEasyRequest with this = " << (void*)this); }
// Returns true if this is a base class of ThreadSafeBufferedCurlEasyRequest.
virtual bool isBuffered(void) const { return false; }
ThreadSafeBufferedCurlEasyRequest(void) : mReferenceCount(0)
{ new (ptr()) BufferedCurlEasyRequest;
Dout(dc::curl, "Creating ThreadSafeBufferedCurlEasyRequest with this = " << (void*)this);
AICurlInterface::Stats::ThreadSafeBufferedCurlEasyRequest_count++; }
~ThreadSafeBufferedCurlEasyRequest()
{ Dout(dc::curl, "Destructing ThreadSafeBufferedCurlEasyRequest with this = " << (void*)this);
--AICurlInterface::Stats::ThreadSafeBufferedCurlEasyRequest_count; }
private:
LLAtomicU32 mReferenceCount;
friend void intrusive_ptr_add_ref(ThreadSafeCurlEasyRequest* p); // Called by boost::intrusive_ptr when a new copy of a boost::intrusive_ptr<ThreadSafeCurlEasyRequest> is made.
friend void intrusive_ptr_release(ThreadSafeCurlEasyRequest* p); // Called by boost::intrusive_ptr when a boost::intrusive_ptr<ThreadSafeCurlEasyRequest> is destroyed.
};
// Same as the above but adds a CurlResponderBuffer. The latter has its own locking in order to
// allow casting the underlying CurlEasyRequest to ThreadSafeCurlEasyRequest, independent of
// what class it is part of: ThreadSafeCurlEasyRequest or ThreadSafeBufferedCurlEasyRequest.
// The virtual destructor of ThreadSafeCurlEasyRequest allows to treat each easy handle transparently
// as a ThreadSafeCurlEasyRequest object, or optionally dynamic_cast it to a ThreadSafeBufferedCurlEasyRequest.
// Note: the order of these base classes is important: AIThreadSafeSimple<CurlResponderBuffer> is now
// destructed before ThreadSafeCurlEasyRequest is.
class ThreadSafeBufferedCurlEasyRequest : public ThreadSafeCurlEasyRequest, public AIThreadSafeSimple<CurlResponderBuffer> {
public:
// Throws AICurlNoEasyHandle.
ThreadSafeBufferedCurlEasyRequest(void) { new (AIThreadSafeSimple<CurlResponderBuffer>::ptr()) CurlResponderBuffer; }
/*virtual*/ bool isBuffered(void) const { return true; }
friend void intrusive_ptr_add_ref(ThreadSafeBufferedCurlEasyRequest* p); // Called by boost::intrusive_ptr when a new copy of a boost::intrusive_ptr<ThreadSafeBufferedCurlEasyRequest> is made.
friend void intrusive_ptr_release(ThreadSafeBufferedCurlEasyRequest* p); // Called by boost::intrusive_ptr when a boost::intrusive_ptr<ThreadSafeBufferedCurlEasyRequest> is destroyed.
};
// The curl easy request type wrapped in a reference counting pointer.
typedef boost::intrusive_ptr<AICurlPrivate::ThreadSafeCurlEasyRequest> CurlEasyRequestPtr;
typedef boost::intrusive_ptr<ThreadSafeBufferedCurlEasyRequest> BufferedCurlEasyRequestPtr;
// This class wraps CURLM*'s.
// It guarantees that a pointer is cleaned up when no longer needed, as required by libcurl.

1115
indra/llmessage/aicurlthread.cpp
View File

File diff suppressed because it is too large Load Diff

26
indra/llmessage/aicurlthread.h
View File

@@ -59,7 +59,7 @@ class MultiHandle : public CurlMultiHandle
~MultiHandle();
// Add/remove an easy handle to/from a multi session.
CURLMcode add_easy_request(AICurlEasyRequest const& easy_request);
void add_easy_request(AICurlEasyRequest const& easy_request);
CURLMcode remove_easy_request(AICurlEasyRequest const& easy_request, bool as_per_command = false);
// Reads/writes available data from a particular socket (non-blocking).
@@ -73,26 +73,28 @@ class MultiHandle : public CurlMultiHandle
private:
typedef std::set<AICurlEasyRequest, AICurlEasyRequestCompare> addedEasyRequests_type;
addedEasyRequests_type mAddedEasyRequests;
bool mHandleAddedOrRemoved; // Set when an easy handle was added or removed, reset in check_run_count().
int mPrevRunningHandles; // The last value of mRunningHandles that check_run_count() was called with.
int mRunningHandles; // The last value returned by curl_multi_socket_action.
long mTimeOut; // The last time out in ms as set by the callback CURLMOPT_TIMERFUNCTION.
addedEasyRequests_type mAddedEasyRequests; // All easy requests currently added to the multi handle.
long mTimeout; // The last timeout in ms as set by the callback CURLMOPT_TIMERFUNCTION.
private:
// Store result and trigger events for easy request.
void finish_easy_request(AICurlEasyRequest const& easy_request, CURLcode result);
// Remove easy request at iter (must exist).
// Note that it's possible that a new request from a PerHostRequestQueue::mQueuedRequests is inserted before iter.
CURLMcode remove_easy_request(addedEasyRequests_type::iterator const& iter, bool as_per_command);
static int socket_callback(CURL* easy, curl_socket_t s, int action, void* userp, void* socketp);
static int timer_callback(CURLM* multi, long timeout_ms, void* userp);
public:
// Returns the number of active easy handles as reported by the last call to curl_multi_socket_action.
int getRunningHandles(void) const { return mRunningHandles; }
// Returns how long to wait for socket action before calling socket_action(CURL_SOCKET_TIMEOUT, 0), in ms.
int getTimeOut(void) const { return mTimeOut; }
int getTimeout(void) const { return mTimeout; }
// This is called before sleeping, after calling (one or more times) socket_action.
void check_run_count(void);
void check_msg_queue(void);
// Called from the main loop every time select() timed out.
void handle_stalls(void);
public:
//-----------------------------------------------------------------------------

153
indra/llmessage/aihttpheaders.cpp Normal file
View File

@@ -0,0 +1,153 @@
/**
* @file aihttpheaders.cpp
* @brief Implementation of AIHTTPHeaders
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 15/08/2012
* Initial version, written by Aleric Inglewood @ SL
*/
#include "sys.h"
#include "aihttpheaders.h"
#include <curl/curl.h>
#ifdef DEBUG_CURLIO
#include "debug_libcurl.h"
#endif
AIHTTPHeaders::AIHTTPHeaders(std::string const& key, std::string const& value) : mContainer(new Container)
{
addHeader(key, value);
}
bool AIHTTPHeaders::addHeader(std::string const& key, std::string const& value, op_type op)
{
if (!mContainer)
{
mContainer = new Container;
}
insert_t res = mContainer->mKeyValuePairs.insert(container_t::value_type(key, value));
bool key_already_exists = !res.second;
if (key_already_exists)
{
llassert_always(op != new_header);
if (op == replace_if_exists)
res.first->second = value;
}
return key_already_exists;
}
void AIHTTPHeaders::append_to(curl_slist*& slist) const
{
if (!mContainer)
return;
container_t::const_iterator const end = mContainer->mKeyValuePairs.end();
for (container_t::const_iterator iter = mContainer->mKeyValuePairs.begin(); iter != end; ++iter)
{
slist = curl_slist_append(slist, llformat("%s: %s", iter->first.c_str(), iter->second.c_str()).c_str());
}
}
bool AIHTTPHeaders::hasHeader(std::string const& key) const
{
return !mContainer ? false : (mContainer->mKeyValuePairs.find(key) != mContainer->mKeyValuePairs.end());
}
bool AIHTTPHeaders::getValue(std::string const& key, std::string& value_out) const
{
AIHTTPHeaders::container_t::const_iterator iter;
if (!mContainer || (iter = mContainer->mKeyValuePairs.find(key)) == mContainer->mKeyValuePairs.end())
return false;
value_out = iter->second;
return true;
}
std::ostream& operator<<(std::ostream& os, AIHTTPHeaders const& headers)
{
os << '{';
if (headers.mContainer)
{
bool first = true;
AIHTTPHeaders::container_t::const_iterator const end = headers.mContainer->mKeyValuePairs.end();
for (AIHTTPHeaders::container_t::const_iterator iter = headers.mContainer->mKeyValuePairs.begin(); iter != end; ++iter)
{
if (!first)
os << ", ";
os << '"' << iter->first << ": " << iter->second << '"';
first = false;
}
}
os << '}';
return os;
}
void AIHTTPReceivedHeaders::addHeader(std::string const& key, std::string const& value)
{
if (!mContainer)
{
mContainer = new Container;
}
mContainer->mKeyValuePairs.insert(container_t::value_type(key, value));
}
bool AIHTTPReceivedHeaders::hasHeader(std::string const& key) const
{
return !mContainer ? false : (mContainer->mKeyValuePairs.find(key) != mContainer->mKeyValuePairs.end());
}
bool AIHTTPReceivedHeaders::getFirstValue(std::string const& key, std::string& value_out) const
{
AIHTTPReceivedHeaders::container_t::const_iterator iter;
if (!mContainer || (iter = mContainer->mKeyValuePairs.find(key)) == mContainer->mKeyValuePairs.end())
return false;
value_out = iter->second;
return true;
}
bool AIHTTPReceivedHeaders::getValues(std::string const& key, range_type& value_out) const
{
if (!mContainer)
return false;
value_out = mContainer->mKeyValuePairs.equal_range(key);
return value_out.first != value_out.second;
}
std::ostream& operator<<(std::ostream& os, AIHTTPReceivedHeaders const& headers)
{
os << '{';
if (headers.mContainer)
{
bool first = true;
AIHTTPReceivedHeaders::container_t::const_iterator const end = headers.mContainer->mKeyValuePairs.end();
for (AIHTTPReceivedHeaders::container_t::const_iterator iter = headers.mContainer->mKeyValuePairs.begin(); iter != end; ++iter)
{
if (!first)
os << ", ";
os << '"' << iter->first << ": " << iter->second << '"';
first = false;
}
}
os << '}';
return os;
}

158
indra/llmessage/aihttpheaders.h Normal file
View File

@@ -0,0 +1,158 @@
/**
* @file aihttpheaders.h
* @brief Keep a list of HTTP headers.
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 15/08/2012
* Initial version, written by Aleric Inglewood @ SL
* 21/10/2012
* Added AIHTTPReceivedHeaders
*/
#ifndef AIHTTPHEADERS_H
#define AIHTTPHEADERS_H
#include <string>
#include <map>
#include <iosfwd>
#include <algorithm>
#include "llpointer.h"
#include "llthread.h" // LLThreadSafeRefCount
extern "C" struct curl_slist;
class AIHTTPHeaders {
public:
enum op_type
{
new_header, // The inserted header must be the first one.
replace_if_exists, // If a header of this type already exists, replace it. Otherwise add the header.
keep_existing_header // If a header of this type already exists, do nothing.
};
// Construct an empty container.
AIHTTPHeaders(void) { }
// Construct a container with a single header.
AIHTTPHeaders(std::string const& key, std::string const& value);
// Clear all headers.
void clear(void) { if (mContainer) mContainer->mKeyValuePairs.clear(); }
// Add a header. Returns true if the header already existed.
bool addHeader(std::string const& key, std::string const& value, op_type op = new_header);
// Return true if there are no headers associated with this object.
bool empty(void) const { return !mContainer || mContainer->mKeyValuePairs.empty(); }
// Return true if the header already exists.
bool hasHeader(std::string const& key) const;
// Return true if key exists and fill value_out with the value. Return false otherwise.
bool getValue(std::string const& key, std::string& value_out) const;
// Append the headers to slist.
void append_to(curl_slist*& slist) const;
// For debug purposes.
friend std::ostream& operator<<(std::ostream& os, AIHTTPHeaders const& headers);
private:
typedef std::map<std::string, std::string> container_t;
typedef std::pair<container_t::iterator, bool> insert_t;
struct Container : public LLThreadSafeRefCount {
container_t mKeyValuePairs;
};
LLPointer<Container> mContainer;
};
// Functor that returns true if c1 is less than c2, ignoring bit 5.
// The effect is that characters in the range a-z equivalent ordering with A-Z.
// This function assumes UTF-8 or ASCII encoding!
//
// Note that other characters aren't important in the case of HTTP header keys;
// however if one considers all printable ASCII characters, then this functor
// also compares "@[\]^" equal to "`{|}~" (any other is either not printable or
// would be equal to a not printable character).
struct AIHTTPReceivedHeadersCharCompare {
bool operator()(std::string::value_type c1, std::string::value_type c2) const
{
static std::string::value_type const bit5 = 0x20;
return (c1 | bit5) < (c2 | bit5);
}
};
// Functor to lexiographically compare two HTTP header keys using the above predicate.
// This means that for example "Content-Type" and "content-type" will have equivalent ordering.
struct AIHTTPReceivedHeadersCompare {
bool operator()(std::string const& h1, std::string const& h2) const
{
static AIHTTPReceivedHeadersCharCompare predicate;
return std::lexicographical_compare(h1.begin(), h1.end(), h2.begin(), h2.end(), predicate);
}
};
class AIHTTPReceivedHeaders {
private:
typedef std::multimap<std::string, std::string, AIHTTPReceivedHeadersCompare> container_t;
public:
typedef container_t::const_iterator iterator_type;
typedef std::pair<iterator_type, iterator_type> range_type;
// Construct an empty container.
AIHTTPReceivedHeaders(void) { }
// Clear all headers.
void clear(void) { if (mContainer) mContainer->mKeyValuePairs.clear(); }
// Add a header.
void addHeader(std::string const& key, std::string const& value);
// Return true if there are no headers associated with this object.
bool empty(void) const { return !mContainer || mContainer->mKeyValuePairs.empty(); }
// Return true if the header exists.
bool hasHeader(std::string const& key) const;
// Return true if key exists and fill value_out with the value. Return false otherwise.
bool getFirstValue(std::string const& key, std::string& value_out) const;
// Return true if key exists and fill value_out with all values. Return false otherwise.
bool getValues(std::string const& key, range_type& value_out) const;
// For debug purposes.
friend std::ostream& operator<<(std::ostream& os, AIHTTPReceivedHeaders const& headers);
private:
struct Container : public LLThreadSafeRefCount {
container_t mKeyValuePairs;
};
LLPointer<Container> mContainer;
};
#endif // AIHTTPHEADERS_H

895
indra/llmessage/aihttptimeoutpolicy.cpp Normal file
View File

@@ -0,0 +1,895 @@
/**
* @file aihttptimeoutpolicy.cpp
* @brief Implementation of AIHTTPTimeoutPolicy
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 24/08/2012
* Initial version, written by Aleric Inglewood @ SL
*/
#include "sys.h"
#include "aihttptimeoutpolicy.h"
#define NOMINMAX
#include "llerror.h"
#include "lldefs.h"
#include "v3math.h"
#include <vector>
//!
// Timing of a HTML connection.
//
// Request call
// |
// v ... <--low speed time--> ... ... <--low speed time--> ...
// <--request queued--><--DNS lookup--><--connect margin--><--data transfer to server--><--reply delay--><--data transfer from server-->
// <------------------------------------------curl transaction----------------------------------------------------->
// <--------------------------------------------------------------total delay---------------------------------------------------------->
// |
// v
// finished
// For now, low speed limit is the same for up and download: usually download is (much) higher, but we have to take into account that
// there might also be multiple downloads at the same time, more than simultaneous uploads.
// Absolute maxima (min/max range):
// These values are intuitively determined and rather arbitrary.
namespace {
U16 const ABS_min_DNS_lookup = 0; // Rationale: If a FAST roundtrip is demanded, then setting the DNS lookup grace time
// at 0 seconds will not make a connection fail when the lookup takes 1 second, it
// just means that no EXTRA time is added to the connect time.
U16 const ABS_max_DNS_lookup = 300; // Waiting longer than 5 minutes never makes sense.
U16 const ABS_min_connect_time = 1; // Can't demand 0 seconds, and we deal with integer numbers here.
U16 const ABS_max_connect_time = 30; // Making a TCP/IP connection should REALLY succeed within 30 seconds or we rather try again.
U16 const ABS_min_reply_delay = 1; // Can't demand 0 seconds, and we deal with integer numbers here.
U16 const ABS_max_reply_delay = 120; // If the server needs more than 2 minutes to find the reply then something just HAS to be wrong :/.
U16 const ABS_min_low_speed_time = 4; // Intuitively, I think it makes no sense to average a download speed over less than 4 seconds.
U16 const ABS_max_low_speed_time = 120; // Averaging it over a time considerably larger than the normal timeout periods makes no sense either.
U32 const ABS_min_low_speed_limit = 1; // In case you don't want to timeout when there is any data received at all.
U32 const ABS_max_low_speed_limit = 1000000; // This limit is almost certainly higher than the maximum speed you get from the server.
U16 const ABS_min_transaction = 60; // This is an absurd low value for experimentation. In reality, you should control
// termination of really slow connections through the low_speed settings.
U16 const ABS_max_transaction = 1200; // Insane long time. Downloads a texture of 4 MB at 3.5 kB/s. Textures are compressed though ;).
U16 const ABS_min_total_delay = 60; // This is an absurd low value for experimentation. In reality, you should control
// termination of really slow connections through the low_speed settings.
U16 const ABS_max_total_delay = 3000; // Insane long time, for when someone wants to be ABSOLUTELY sure this isn't the bottleneck.
using namespace AIHTTPTimeoutPolicyOperators;
// Default policy values.
U16 const AITP_default_DNS_lookup_grace = 60; // Allow for 60 seconds long DNS look ups.
U16 const AITP_default_maximum_connect_time = 10; // Allow the SSL/TLS connection through a proxy, including handshakes, to take up to 10 seconds.
U16 const AITP_default_maximum_reply_delay = 60; // Allow the server 60 seconds to do whatever it has to do before starting to send data.
U16 const AITP_default_low_speed_time = 30; // If a transfer speed drops below AITP_default_low_speed_limit bytes/s for 30 seconds, terminate the transfer.
U32 const AITP_default_low_speed_limit = 56000; // In bytes per second (use for CURLOPT_LOW_SPEED_LIMIT).
U16 const AITP_default_maximum_curl_transaction = 300; // Allow large files to be transfered over slow connections.
U16 const AITP_default_maximum_total_delay = 600; // Avoid "leaking" by terminating anything that wasn't completed after 10 minutes.
} // namespace
AIHTTPTimeoutPolicy& AIHTTPTimeoutPolicy::operator=(AIHTTPTimeoutPolicy const& rhs)
{
// You're not allowed to assign to a policy that is based on another policy.
llassert(!mBase);
mDNSLookupGrace = rhs.mDNSLookupGrace;
mMaximumConnectTime = rhs.mMaximumConnectTime;
mMaximumReplyDelay = rhs.mMaximumReplyDelay;
mLowSpeedTime = rhs.mLowSpeedTime;
mLowSpeedLimit = rhs.mLowSpeedLimit;
mMaximumCurlTransaction = rhs.mMaximumCurlTransaction;
mMaximumTotalDelay = rhs.mMaximumTotalDelay;
return *this;
}
AIHTTPTimeoutPolicy::AIHTTPTimeoutPolicy(char const* name,
U16 dns_lookup_grace, U16 subsequent_connects, U16 reply_delay,
U16 low_speed_time, U32 low_speed_limit,
U16 curl_transaction, U16 total_delay) :
mName(name),
mBase(NULL),
mDNSLookupGrace(dns_lookup_grace),
mMaximumConnectTime(subsequent_connects),
mMaximumReplyDelay(reply_delay),
mLowSpeedTime(low_speed_time),
mLowSpeedLimit(low_speed_limit),
mMaximumCurlTransaction(curl_transaction),
mMaximumTotalDelay(total_delay)
{
sanity_checks();
}
struct PolicyOp {
PolicyOp* mNext;
PolicyOp(void) : mNext(NULL) { }
PolicyOp(PolicyOp& op) : mNext(&op) { }
virtual void perform(AIHTTPTimeoutPolicy* policy) const { }
void nextOp(AIHTTPTimeoutPolicy* policy) const { if (mNext) mNext->perform(policy); }
};
class AIHTTPTimeoutPolicyBase : public AIHTTPTimeoutPolicy {
private:
std::vector<AIHTTPTimeoutPolicy*> mDerived; // Policies derived from this one.
public:
AIHTTPTimeoutPolicyBase(U16 dns_lookup_grace, U16 subsequent_connects, U16 reply_delay,
U16 low_speed_time, U32 low_speed_limit,
U16 curl_transaction, U16 total_delay) :
AIHTTPTimeoutPolicy(NULL, dns_lookup_grace, subsequent_connects, reply_delay, low_speed_time, low_speed_limit, curl_transaction, total_delay) { }
// Derive base from base.
AIHTTPTimeoutPolicyBase(AIHTTPTimeoutPolicyBase& rhs, PolicyOp const& op) : AIHTTPTimeoutPolicy(rhs) { op.perform(this); }
// Called for every derived policy.
void derived(AIHTTPTimeoutPolicy* derived) { mDerived.push_back(derived); }
// Provide public acces to sDebugSettingsCurlTimeout for this compilation unit.
static AIHTTPTimeoutPolicyBase& getDebugSettingsCurlTimeout(void) { return sDebugSettingsCurlTimeout; }
protected:
friend void AIHTTPTimeoutPolicy::setDefaultCurlTimeout(AIHTTPTimeoutPolicy const& timeout);
AIHTTPTimeoutPolicyBase& operator=(AIHTTPTimeoutPolicy const& rhs);
};
AIHTTPTimeoutPolicy::AIHTTPTimeoutPolicy(AIHTTPTimeoutPolicy& base) :
mName(NULL),
mBase(static_cast<AIHTTPTimeoutPolicyBase*>(&base)),
mDNSLookupGrace(base.mDNSLookupGrace),
mMaximumConnectTime(base.mMaximumConnectTime),
mMaximumReplyDelay(base.mMaximumReplyDelay),
mLowSpeedTime(base.mLowSpeedTime),
mLowSpeedLimit(base.mLowSpeedLimit),
mMaximumCurlTransaction(base.mMaximumCurlTransaction),
mMaximumTotalDelay(base.mMaximumTotalDelay)
{
}
AIHTTPTimeoutPolicyBase& AIHTTPTimeoutPolicyBase::operator=(AIHTTPTimeoutPolicy const& rhs)
{
AIHTTPTimeoutPolicy::operator=(rhs);
return *this;
}
AIHTTPTimeoutPolicy::AIHTTPTimeoutPolicy(char const* name, AIHTTPTimeoutPolicyBase& base) :
mName(name),
mBase(&base),
mDNSLookupGrace(mBase->mDNSLookupGrace),
mMaximumConnectTime(mBase->mMaximumConnectTime),
mMaximumReplyDelay(mBase->mMaximumReplyDelay),
mLowSpeedTime(mBase->mLowSpeedTime),
mLowSpeedLimit(mBase->mLowSpeedLimit),
mMaximumCurlTransaction(mBase->mMaximumCurlTransaction),
mMaximumTotalDelay(mBase->mMaximumTotalDelay)
{
sNameMap.insert(namemap_t::value_type(name, this));
// Register for changes to the base policy.
mBase->derived(this);
}
//static
void AIHTTPTimeoutPolicy::setDefaultCurlTimeout(AIHTTPTimeoutPolicy const& timeout)
{
sDebugSettingsCurlTimeout = timeout;
llinfos << "CurlTimeout Debug Settings now"
": DNSLookup: " << sDebugSettingsCurlTimeout.mDNSLookupGrace <<
"; Connect: " << sDebugSettingsCurlTimeout.mMaximumConnectTime <<
"; ReplyDelay: " << sDebugSettingsCurlTimeout.mMaximumReplyDelay <<
"; LowSpeedTime: " << sDebugSettingsCurlTimeout.mLowSpeedTime <<
"; LowSpeedLimit: " << sDebugSettingsCurlTimeout.mLowSpeedLimit <<
"; MaxTransaction: " << sDebugSettingsCurlTimeout.mMaximumCurlTransaction <<
"; MaxTotalDelay: " << sDebugSettingsCurlTimeout.mMaximumTotalDelay << llendl;
if (sDebugSettingsCurlTimeout.mDNSLookupGrace < AITP_default_DNS_lookup_grace)
{
llwarns << "CurlTimeoutDNSLookup (" << sDebugSettingsCurlTimeout.mDNSLookupGrace << ") is lower than the built-in default value (" << AITP_default_DNS_lookup_grace << ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mMaximumConnectTime < AITP_default_maximum_connect_time)
{
llwarns << "CurlTimeoutConnect (" << sDebugSettingsCurlTimeout.mMaximumConnectTime << ") is lower than the built-in default value (" << AITP_default_maximum_connect_time << ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mMaximumReplyDelay < AITP_default_maximum_reply_delay)
{
llwarns << "CurlTimeoutReplyDelay (" << sDebugSettingsCurlTimeout.mMaximumReplyDelay << ") is lower than the built-in default value (" << AITP_default_maximum_reply_delay << ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mLowSpeedTime < AITP_default_low_speed_time)
{
llwarns << "CurlTimeoutLowSpeedTime (" << sDebugSettingsCurlTimeout.mLowSpeedTime << ") is lower than the built-in default value (" << AITP_default_low_speed_time << ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mLowSpeedLimit > AITP_default_low_speed_limit)
{
llwarns << "CurlTimeoutLowSpeedLimit (" << sDebugSettingsCurlTimeout.mLowSpeedLimit << ") is higher than the built-in default value (" << AITP_default_low_speed_limit << ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mMaximumCurlTransaction < AITP_default_maximum_curl_transaction)
{
llwarns << "CurlTimeoutMaxTransaction (" << sDebugSettingsCurlTimeout.mMaximumCurlTransaction << ") is lower than the built-in default value (" << AITP_default_maximum_curl_transaction<< ")." << llendl;
}
if (sDebugSettingsCurlTimeout.mMaximumTotalDelay < AITP_default_maximum_total_delay)
{
llwarns << "CurlTimeoutMaxTotalDelay (" << sDebugSettingsCurlTimeout.mMaximumTotalDelay << ") is lower than the built-in default value (" << AITP_default_maximum_total_delay << ")." << llendl;
}
}
//static
AIHTTPTimeoutPolicy const& AIHTTPTimeoutPolicy::getDebugSettingsCurlTimeout(void)
{
return sDebugSettingsCurlTimeout;
}
#ifdef SHOW_ASSERT
#include "aithreadid.h"
static AIThreadID curlthread(AIThreadID::none); // Initialized by getConnectTimeout.
#endif
static std::set<std::string> gSeenHostnames;
U16 AIHTTPTimeoutPolicy::getConnectTimeout(std::string const& hostname) const
{
#ifdef SHOW_ASSERT
// Only the CURL-THREAD may access gSeenHostnames.
if (curlthread.is_no_thread())
curlthread.reset();
llassert(curlthread.equals_current_thread());
#endif
U16 connect_timeout = mMaximumConnectTime;
// Add the hostname to the list of seen hostnames, if not already there.
if (gSeenHostnames.insert(hostname).second)
connect_timeout += mDNSLookupGrace; // If the host is not in the list, increase the connect timeout with mDNSLookupGrace.
return connect_timeout;
}
//static
bool AIHTTPTimeoutPolicy::connect_timed_out(std::string const& hostname)
{
llassert(curlthread.equals_current_thread());
// This is called when a connect to hostname timed out on connect.
// If the hostname is currently in the list, remove it and return true
// so that subsequent connects will get more time to connect.
// Otherwise return false.
return gSeenHostnames.erase(hostname) > 0;
}
//=======================================================================================================
// Start of policy operation definitions.
namespace AIHTTPTimeoutPolicyOperators {
// Note: Policies are applied in the order First(x, Second(y, Third(z))) etc,
// where the last (Third) has the highest priority.
// For example: Transaction(5, Connect(40)) would first enforce a transaction time of 5 seconds,
// and then a connect time of 40 seconds, even if that would mean increasing the transaction
// time again.
struct DNS : PolicyOp {
int mSeconds;
DNS(int seconds) : mSeconds(seconds) { }
DNS(int seconds, PolicyOp& op) : PolicyOp(op), mSeconds(seconds) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(void) { return ABS_min_DNS_lookup; }
static U16 max(void) { return ABS_max_DNS_lookup; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
struct Connect : PolicyOp {
int mSeconds;
Connect(int seconds) : mSeconds(seconds) { }
Connect(int seconds, PolicyOp& op) : PolicyOp(op), mSeconds(seconds) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(void) { return ABS_min_connect_time; }
static U16 max(void) { return ABS_max_connect_time; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
struct Reply : PolicyOp {
int mSeconds;
Reply(int seconds) : mSeconds(seconds) { }
Reply(int seconds, PolicyOp& op) : PolicyOp(op), mSeconds(seconds) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(void) { return ABS_min_reply_delay; }
static U16 max(void) { return ABS_max_reply_delay; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
struct Speed : PolicyOp {
int mSeconds;
int mRate;
Speed(int seconds, int rate) : mSeconds(seconds), mRate(rate) { }
Speed(int seconds, int rate, PolicyOp& op) : PolicyOp(op), mSeconds(seconds), mRate(rate) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(void) { return ABS_min_low_speed_time; }
static U16 max(AIHTTPTimeoutPolicy const* policy) { return llmin(ABS_max_low_speed_time, (U16)(policy->mMaximumCurlTransaction / 2)); }
static U32 lmin(void) { return ABS_min_low_speed_limit; }
static U32 lmax(void) { return ABS_max_low_speed_limit; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
struct Transaction : PolicyOp {
int mSeconds;
Transaction(int seconds) : mSeconds(seconds) { }
Transaction(int seconds, PolicyOp& op) : PolicyOp(op), mSeconds(seconds) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(AIHTTPTimeoutPolicy const* policy) { return llmax(ABS_min_transaction, (U16)(policy->mMaximumConnectTime + policy->mMaximumReplyDelay + 4 * policy->mLowSpeedTime)); }
static U16 max(void) { return ABS_max_transaction; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
struct Total : PolicyOp {
int mSeconds;
Total(int seconds) : mSeconds(seconds) { }
Total(int seconds, PolicyOp& op) : PolicyOp(op), mSeconds(seconds) { }
static void fix(AIHTTPTimeoutPolicy* policy);
static U16 min(AIHTTPTimeoutPolicy const* policy) { return llmax(ABS_min_total_delay, (U16)(policy->mMaximumCurlTransaction + 1)); }
static U16 max(void) { return ABS_max_total_delay; }
virtual void perform(AIHTTPTimeoutPolicy* policy) const;
};
void DNS::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mDNSLookupGrace = mSeconds;
fix(policy);
nextOp(policy);
}
void Connect::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mMaximumConnectTime = mSeconds;
fix(policy);
nextOp(policy);
}
void Reply::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mMaximumReplyDelay = mSeconds;
fix(policy);
nextOp(policy);
}
void Speed::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mLowSpeedTime = mSeconds;
policy->mLowSpeedLimit = mRate;
fix(policy);
nextOp(policy);
}
void Transaction::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mMaximumCurlTransaction = mSeconds;
fix(policy);
nextOp(policy);
}
void Total::perform(AIHTTPTimeoutPolicy* policy) const
{
policy->mMaximumTotalDelay = mSeconds;
fix(policy);
nextOp(policy);
}
void DNS::fix(AIHTTPTimeoutPolicy* policy)
{
if (policy->mDNSLookupGrace > max())
{
policy->mDNSLookupGrace = max();
}
else if (policy->mDNSLookupGrace < min())
{
policy->mDNSLookupGrace = min();
}
}
void Connect::fix(AIHTTPTimeoutPolicy* policy)
{
bool changed = false;
if (policy->mMaximumConnectTime > max())
{
policy->mMaximumConnectTime = max();
changed = true;
}
else if (policy->mMaximumConnectTime < min())
{
policy->mMaximumConnectTime = min();
changed = true;
}
if (changed)
{
// Transaction limits depend on Connect.
Transaction::fix(policy);
}
}
void Reply::fix(AIHTTPTimeoutPolicy* policy)
{
bool changed = false;
if (policy->mMaximumReplyDelay > max())
{
policy->mMaximumReplyDelay = max();
changed = true;
}
else if (policy->mMaximumReplyDelay < min())
{
policy->mMaximumReplyDelay = min();
changed = true;
}
if (changed)
{
// Transaction limits depend on Reply.
Transaction::fix(policy);
}
}
void Speed::fix(AIHTTPTimeoutPolicy* policy)
{
bool changed = false;
if (policy->mLowSpeedTime > ABS_max_low_speed_time)
{
policy->mLowSpeedTime = ABS_max_low_speed_time;
changed = true;
}
else if (policy->mLowSpeedTime != 0 && policy->mLowSpeedTime < min())
{
policy->mLowSpeedTime = min();
changed = true;
}
if (changed)
{
// Transaction limits depend on Speed time.
Transaction::fix(policy);
}
if (policy->mLowSpeedTime > max(policy))
{
policy->mLowSpeedTime = max(policy);
}
if (policy->mLowSpeedLimit > lmax())
{
policy->mLowSpeedLimit = lmax();
}
else if (policy->mLowSpeedLimit != 0 && policy->mLowSpeedLimit < lmin())
{
policy->mLowSpeedLimit = lmin();
}
}
void Transaction::fix(AIHTTPTimeoutPolicy* policy)
{
bool changed = false;
if (policy->mMaximumCurlTransaction > max())
{
policy->mMaximumCurlTransaction = max();
changed = true;
}
else if (policy->mMaximumCurlTransaction < ABS_min_transaction)
{
policy->mMaximumCurlTransaction = ABS_min_transaction;
changed = true;
}
if (changed)
{
// Totals minimum limit depends on Transaction.
Total::fix(policy);
// Transaction limits depend on Connect, Reply and Speed time.
if (policy->mMaximumCurlTransaction < min(policy))
{
// We need to achieve the following (from Transaction::min()):
// policy->mMaximumCurlTransaction >= policy->mMaximumConnectTime + policy->mMaximumReplyDelay + 4 * policy->mLowSpeedTime
// There isn't a single way to fix this, so we just do something randomly intuitive.
// We consider the vector space <connect_time, reply_delay, low_speed_time>;
// In other words, we need to compare with the dot product of <1, 1, 4>.
LLVector3 const ref(1, 1, 4);
// The shortest allowed vector is:
LLVector3 const vec_min(ABS_min_connect_time, ABS_min_reply_delay, ABS_min_low_speed_time);
// Initialize the result vector to (0, 0, 0) (in the vector space with shifted origin).
LLVector3 vec_res;
// Check if there is a solution at all:
if (policy->mMaximumCurlTransaction > ref * vec_min) // Is vec_min small enough?
{
// The current point is:
LLVector3 vec_cur(policy->mMaximumConnectTime, policy->mMaximumReplyDelay, policy->mLowSpeedTime);
// The default point is:
LLVector3 vec_def(AITP_default_maximum_connect_time, AITP_default_maximum_reply_delay, AITP_default_low_speed_time);
// Move the origin.
vec_cur -= vec_min;
vec_def -= vec_min;
// Normalize the default vector (we only need it's direction).
vec_def.normalize();
// Project the current vector onto the default vector (dp = default projection):
LLVector3 vec_dp = vec_def * (vec_cur * vec_def);
// Check if the projection is a solution and choose the vectors between which the result lays.
LLVector3 a; // vec_min is too small (a = (0, 0, 0) which corresponds to vec_min).
LLVector3 b = vec_cur; // vec_cur is too large.
if (policy->mMaximumCurlTransaction > ref * (vec_dp + vec_min)) // Is vec_dp small enough too?
{
a = vec_dp; // New lower bound.
}
else
{
b = vec_dp; // New upper bound.
}
// Find vec_res = a + lambda * (b - a), where 0 <= lambda <= 1, such that
// policy->mMaximumCurlTransaction == ref * (vec_res + vec_min).
//
// Note that ref * (b - a) must be non-zero because if it wasn't then changing lambda wouldn't have
// any effect on right-hand side of the equation (ref * (vec_res + vec_min)) which in contradiction
// with the fact that a is a solution and b is not.
F32 lambda = (policy->mMaximumCurlTransaction - ref * (a + vec_min)) / (ref * (b - a));
vec_res = a + lambda * (b - a);
}
// Shift origin back and fill in the result.
vec_res += vec_min;
policy->mMaximumConnectTime = vec_res[VX];
policy->mMaximumReplyDelay = vec_res[VY];
policy->mLowSpeedTime = vec_res[VZ];
}
}
if (policy->mMaximumCurlTransaction < min(policy))
{
policy->mMaximumCurlTransaction = min(policy);
}
}
void Total::fix(AIHTTPTimeoutPolicy* policy)
{
bool changed = false;
if (policy->mMaximumTotalDelay > max())
{
policy->mMaximumTotalDelay = max();
changed = true;
}
else if (policy->mMaximumTotalDelay < ABS_min_total_delay)
{
policy->mMaximumTotalDelay = ABS_min_total_delay;
changed = true;
}
if (changed)
{
// Totals minimum limit depends on Transaction.
// We have to correct mMaximumCurlTransaction such that (from Total::min)
// mMaximumTotalDelay >= llmax((int)ABS_min_total_delay, policy->mMaximumCurlTransaction + 1)
if (policy->mMaximumTotalDelay < policy->mMaximumCurlTransaction + 1)
{
policy->mMaximumCurlTransaction = policy->mMaximumTotalDelay - 1;
}
}
if (policy->mMaximumTotalDelay < min(policy))
{
policy->mMaximumTotalDelay = min(policy);
}
}
} // namespace AIHTTPTimeoutPolicyOperators
void AIHTTPTimeoutPolicy::sanity_checks(void) const
{
// Sanity checks.
llassert( DNS::min() <= mDNSLookupGrace && mDNSLookupGrace <= DNS::max());
llassert( Connect::min() <= mMaximumConnectTime && mMaximumConnectTime <= Connect::max());
llassert( Reply::min() <= mMaximumReplyDelay && mMaximumReplyDelay <= Reply::max());
llassert(mLowSpeedTime == 0 ||
(Speed::min() <= mLowSpeedTime && mLowSpeedTime <= Speed::max(this)));
llassert(mLowSpeedLimit == 0 ||
(Speed::lmin() <= mLowSpeedLimit && mLowSpeedLimit <= Speed::lmax()));
llassert(Transaction::min(this) <= mMaximumCurlTransaction && mMaximumCurlTransaction <= Transaction::max());
llassert( Total::min(this) <= mMaximumTotalDelay && mMaximumTotalDelay <= Total::max());
}
//=======================================================================================================
// Start of policy definitions.
// Policy with hardcoded default values.
AIHTTPTimeoutPolicyBase HTTPTimeoutPolicy_default(
AITP_default_DNS_lookup_grace,
AITP_default_maximum_connect_time,
AITP_default_maximum_reply_delay,
AITP_default_low_speed_time,
AITP_default_low_speed_limit,
AITP_default_maximum_curl_transaction,
AITP_default_maximum_total_delay);
//static. Initialized here, but shortly overwritten by Debug Settings.
AIHTTPTimeoutPolicyBase AIHTTPTimeoutPolicy::sDebugSettingsCurlTimeout(
AITP_default_DNS_lookup_grace,
AITP_default_maximum_connect_time,
AITP_default_maximum_reply_delay,
AITP_default_low_speed_time,
AITP_default_low_speed_limit,
AITP_default_maximum_curl_transaction,
AITP_default_maximum_total_delay);
// Note: Broken compiler doesn't allow as to use temporaries for the Operator ojects,
// so they are instantiated separately.
// This used to be '5 seconds'.
Transaction transactionOp5s(5);
AIHTTPTimeoutPolicyBase transfer_5s(AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout(),
transactionOp5s
);
// This used to be '18 seconds'.
Transaction transactionOp18s(18);
AIHTTPTimeoutPolicyBase transfer_18s(AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout(),
transactionOp18s
);
// This used to be '300 seconds'. We derive this from the hardcoded result so users can't mess with it.
Transaction transactionOp300s(300);
AIHTTPTimeoutPolicyBase transfer_300s(HTTPTimeoutPolicy_default,
transactionOp300s
);
// This used to be a call to setopt(CURLOPT_CONNECTTIMEOUT, 40L) with the remark 'Be a little impatient about establishing connections.'
Connect connectOp40s(40);
AIHTTPTimeoutPolicyBase connect_40s(AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout(),
connectOp40s
);
// End of policy definitions.
//=======================================================================================================
bool validateCurlTimeoutDNSLookup(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::DNS op(new_value);
op.perform(&timeout);
return timeout.getDNSLookup() == new_value;
}
bool handleCurlTimeoutDNSLookup(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::DNS op(newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutConnect(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Connect op(new_value);
op.perform(&timeout);
return timeout.getConnect() == new_value;
}
bool handleCurlTimeoutConnect(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Connect op(newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutReplyDelay(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Reply op(new_value);
op.perform(&timeout);
return timeout.getReplyDelay() == new_value;
}
bool handleCurlTimeoutReplyDelay(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Reply op(newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutLowSpeedLimit(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Speed op(timeout.getLowSpeedTime(), new_value);
op.perform(&timeout);
return timeout.getLowSpeedLimit() == new_value;
}
bool handleCurlTimeoutLowSpeedLimit(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Speed op(timeout.getLowSpeedTime(), newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutLowSpeedTime(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Speed op(new_value, timeout.getLowSpeedLimit());
op.perform(&timeout);
return timeout.getLowSpeedTime() == new_value;
}
bool handleCurlTimeoutLowSpeedTime(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Speed op(newvalue.asInteger(), timeout.getLowSpeedLimit());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutMaxTransaction(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Transaction op(new_value);
op.perform(&timeout);
return timeout.getCurlTransaction() == new_value;
}
bool handleCurlTimeoutMaxTransaction(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Transaction op(newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
bool validateCurlTimeoutMaxTotalDelay(LLSD const& newvalue)
{
U32 new_value = newvalue.asInteger();
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Total op(new_value);
op.perform(&timeout);
return timeout.getTotalDelay() == new_value;
}
bool handleCurlTimeoutMaxTotalDelay(LLSD const& newvalue)
{
AIHTTPTimeoutPolicyBase timeout = AIHTTPTimeoutPolicyBase::getDebugSettingsCurlTimeout();
AIHTTPTimeoutPolicyOperators::Total op(newvalue.asInteger());
op.perform(&timeout);
AIHTTPTimeoutPolicy::setDefaultCurlTimeout(timeout);
return true;
}
//static
AIHTTPTimeoutPolicy::namemap_t AIHTTPTimeoutPolicy::sNameMap;
//static
AIHTTPTimeoutPolicy const* AIHTTPTimeoutPolicy::getTimeoutPolicyByName(std::string const& name)
{
namemap_t::iterator iter = sNameMap.find(name);
if (iter == sNameMap.end())
{
if (!name.empty())
{
llwarns << "Cannot find AIHTTPTimeoutPolicy with name \"" << name << "\"." << llendl;
}
return &sDebugSettingsCurlTimeout;
}
return iter->second;
}
//=======================================================================================================
// Start of Responder timeout policy list.
// Note: to find the actual responder class back, search for the name listed here but with upper case first character.
// For example, if the actual responder class is LLAccountingCostResponder then the name used here is accountingCostResponder.
#undef P
#define P(n) AIHTTPTimeoutPolicy n##_timeout(#n)
#define P2(n, b) AIHTTPTimeoutPolicy n##_timeout(#n, b)
// Policy name Policy
P(accountingCostResponder);
P(agentStateResponder);
P(assetUploadResponder);
P(asyncConsoleResponder);
P(authHandler);
P(avatarNameResponder);
P2(baseCapabilitiesComplete, transfer_18s);
P(blockingLLSDPost);
P(blockingLLSDGet);
P(blockingRawGet);
P(charactersResponder);
P(classifiedStatsResponder);
P(consoleResponder);
P2(crashLoggerResponder, transfer_5s);
P(createInventoryCategoryResponder);
P(emeraldDicDownloader);
P(environmentApplyResponder);
P(environmentRequestResponder);
P(estateChangeInfoResponder);
P(eventPollResponder);
P(fetchInventoryResponder);
P(fnPtrResponder);
P2(groupProposalBallotResponder, transfer_300s);
P(homeLocationResponder);
P(HTTPGetResponder);
P(iamHereLogin);
P(iamHere);
P(iamHereVoice);
P2(inventoryModelFetchDescendentsResponder, transfer_300s);
P(inventoryModelFetchItemResponder);
P(lcl_responder);
P(MPImportGetResponder);
P(MPImportPostResponder);
P(mapLayerResponder);
P(mediaTypeResponder);
P(meshDecompositionResponder);
P(meshHeaderResponder);
P(meshLODResponder);
P(meshPhysicsShapeResponder);
P(meshSkinInfoResponder);
P(mimeDiscoveryResponder);
P(moderationModeResponder);
P(muteTextResponder);
P(muteVoiceResponder);
P(navMeshRebakeResponder);
P(navMeshResponder);
P(navMeshStatusResponder);
P(newAgentInventoryVariablePriceResponder);
P(objectCostResponder);
P(objectLinksetsResponder);
P(physicsFlagsResponder);
P(placeAvatarTeleportResponder);
P(productInfoRequestResponder);
P(regionResponder);
P(remoteParcelRequestResponder);
P(responderIgnore);
P(sessionInviteResponder);
P(setDisplayNameResponder);
P2(simulatorFeaturesReceived, transfer_18s);
P(startConferenceChatResponder);
P2(startGroupVoteResponder, transfer_300s);
P(terrainLinksetsResponder);
P(translationReceiver);
P(uploadModelPremissionsResponder);
P(userReportResponder);
P(verifiedDestinationResponder);
P(viewerChatterBoxInvitationAcceptResponder);
P(viewerMediaOpenIDResponder);
P(viewerMediaWebProfileResponder);
P(viewerStatsResponder);
P(viewerVoiceAccountProvisionResponder);
P(voiceCallCapResponder);
P(voiceClientCapResponder);
P(wholeModelFeeResponder);
P(wholeModelUploadResponder);
P2(XMLRPCResponder, connect_40s);

138
indra/llmessage/aihttptimeoutpolicy.h Normal file
View File

@@ -0,0 +1,138 @@
/**
* @file aihttptimeoutpolicy.h
* @brief Store the policy on timing out a HTTP curl transaction.
*
* Copyright (c) 2012, Aleric Inglewood.
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*
* There are special exceptions to the terms and conditions of the GPL as
* it is applied to this Source Code. View the full text of the exception
* in the file doc/FLOSS-exception.txt in this software distribution.
*
* CHANGELOG
* and additional copyright holders.
*
* 24/09/2012
* Initial version, written by Aleric Inglewood @ SL
*/
#ifndef AIHTTPTIMEOUTPOLICY_H
#define AIHTTPTIMEOUTPOLICY_H
#include "stdtypes.h"
#include <string>
#include <map>
class AIHTTPTimeoutPolicyBase;
namespace AIHTTPTimeoutPolicyOperators {
struct DNS;
struct Connect;
struct Reply;
struct Speed;
struct Transaction;
struct Total;
} // namespace AIHTTPTimeoutPolicyOperators
class AIHTTPTimeoutPolicy {
protected:
char const* const mName; // The name of this policy, for debugging purposes.
AIHTTPTimeoutPolicyBase* const mBase; // Policy this policy was based on.
static AIHTTPTimeoutPolicyBase sDebugSettingsCurlTimeout; // CurlTimeout* debug settings.
typedef std::map<std::string, AIHTTPTimeoutPolicy*> namemap_t; // Type of sNameMap.
static namemap_t sNameMap; // Map of name of timeout policies (as returned by name()) to AIHTTPTimeoutPolicy* (this).
private:
U16 mDNSLookupGrace; // Extra connect timeout the first time we connect to a host (this is to allow for DNS lookups).
U16 mMaximumConnectTime; // Connect timeouts any subsequent connects to the same host, assuming the DNS will be cached now.
U16 mMaximumReplyDelay; // Timeout for the period between sending data to the server and the HTTP header of the reply.
U16 mLowSpeedTime; // The time in seconds that a transfer should be below mLowSpeedLimit before to consider it too slow and abort.
U32 mLowSpeedLimit; // Transfer speed in bytes per second that a transfer should be below during mLowSpeedTime seconds to consider it too slow and abort.
U16 mMaximumCurlTransaction; // Timeout for the whole curl transaction (including connect and DNS lookup).
U16 mMaximumTotalDelay; // Timeout from moment of request (including the time a request is/was queued).
friend struct AIHTTPTimeoutPolicyOperators::DNS;
friend struct AIHTTPTimeoutPolicyOperators::Connect;
friend struct AIHTTPTimeoutPolicyOperators::Reply;
friend struct AIHTTPTimeoutPolicyOperators::Speed;
friend struct AIHTTPTimeoutPolicyOperators::Transaction;
friend struct AIHTTPTimeoutPolicyOperators::Total;
public:
// Construct a HTTP timeout policy object that mimics base, or Debug Settings if none given.
AIHTTPTimeoutPolicy(
char const* name,
AIHTTPTimeoutPolicyBase& base = sDebugSettingsCurlTimeout);
// Construct a HTTP timeout policy with exact specifications.
AIHTTPTimeoutPolicy(
char const* name,
U16 dns_lookup_grace,
U16 subsequent_connects,
U16 reply_delay,
U16 low_speed_time,
U32 low_speed_limit,
U16 curl_transaction,
U16 total_delay);
void sanity_checks(void) const;
// Accessors.
char const* name(void) const { return mName; }
U16 getConnectTimeout(std::string const& hostname) const;
U16 getDNSLookup(void) const { return mDNSLookupGrace; }
U16 getConnect(void) const { return mMaximumConnectTime; }
U16 getReplyDelay(void) const { return mMaximumReplyDelay; }
U16 getLowSpeedTime(void) const { return mLowSpeedTime; }
U32 getLowSpeedLimit(void) const { return mLowSpeedLimit; }
U16 getCurlTransaction(void) const { return mMaximumCurlTransaction; }
U16 getTotalDelay(void) const { return mMaximumTotalDelay; }
static AIHTTPTimeoutPolicy const& getDebugSettingsCurlTimeout(void);
static AIHTTPTimeoutPolicy const* getTimeoutPolicyByName(std::string const& name);
// Called once at start up of viewer to set a different default timeout policy than HTTPTimeoutPolicy_default.
static void setDefaultCurlTimeout(AIHTTPTimeoutPolicy const& defaultCurlTimeout);
// Called when a connect to a hostname timed out.
static bool connect_timed_out(std::string const& hostname);
protected:
// Used by AIHTTPTimeoutPolicyBase::AIHTTPTimeoutPolicyBase(AIHTTPTimeoutPolicyBase&).
AIHTTPTimeoutPolicy(AIHTTPTimeoutPolicy&);
// Abused assigned operator (called by AIHTTPTimeoutPolicyBase::operator=).
AIHTTPTimeoutPolicy& operator=(AIHTTPTimeoutPolicy const&);
};
class LLSD;
// Handlers for Debug Setting changes.
bool validateCurlTimeoutDNSLookup(LLSD const& newvalue);
bool handleCurlTimeoutDNSLookup(LLSD const& newvalue);
bool validateCurlTimeoutConnect(LLSD const& newvalue);
bool handleCurlTimeoutConnect(LLSD const& newvalue);
bool validateCurlTimeoutReplyDelay(LLSD const& newvalue);
bool handleCurlTimeoutReplyDelay(LLSD const& newvalue);
bool validateCurlTimeoutLowSpeedLimit(LLSD const& newvalue);
bool handleCurlTimeoutLowSpeedLimit(LLSD const& newvalue);
bool validateCurlTimeoutLowSpeedTime(LLSD const& newvalue);
bool handleCurlTimeoutLowSpeedTime(LLSD const& newvalue);
bool validateCurlTimeoutMaxTransaction(LLSD const& newvalue);
bool handleCurlTimeoutMaxTransaction(LLSD const& newvalue);
bool validateCurlTimeoutMaxTotalDelay(LLSD const& newvalue);
bool handleCurlTimeoutMaxTotalDelay(LLSD const& newvalue);
#endif // AIHTTPTIMEOUTPOLICY_H

26
indra/llmessage/debug_libcurl.cpp
View File

@@ -249,7 +249,7 @@ std::ostream& operator<<(std::ostream& os, CURLoption option)
CASEPRINT(CURLOPT_LOW_SPEED_TIME);
CASEPRINT(CURLOPT_RESUME_FROM);
CASEPRINT(CURLOPT_COOKIE);
CASEPRINT(CURLOPT_RTSPHEADER);
CASEPRINT(CURLOPT_HTTPHEADER);
CASEPRINT(CURLOPT_HTTPPOST);
CASEPRINT(CURLOPT_SSLCERT);
CASEPRINT(CURLOPT_KEYPASSWD);
@@ -548,7 +548,7 @@ char* debug_curl_easy_escape(CURL* curl, char* url, int length)
{
char* ret;
ret = curl_easy_escape(curl, url, length);
Dout(dc::curl, "curl_easy_escape(" << curl << ", \"" << url << "\", " << length << ") = \"" << ret << '"');
Dout(dc::curl, "curl_easy_escape(" << (AICURL*)curl << ", \"" << url << "\", " << length << ") = \"" << ret << '"');
return ret;
}
@@ -569,23 +569,23 @@ CURLcode debug_curl_easy_getinfo(CURL* curl, CURLINFO info, ...)
ret = curl_easy_getinfo(curl, info, param.some_ptr);
if (info == CURLINFO_PRIVATE)
{
Dout(dc::curl, "curl_easy_getinfo(" << curl << ", " << info << ", 0x" << std::hex << (size_t)param.some_ptr << std::dec << ") = " << ret);
Dout(dc::curl, "curl_easy_getinfo(" << (AICURL*)curl << ", " << info << ", 0x" << std::hex << (size_t)param.some_ptr << std::dec << ") = " << ret);
}
else
{
switch((info & CURLINFO_TYPEMASK))
{
case CURLINFO_STRING:
Dout(dc::curl, "curl_easy_getinfo(" << curl << ", " << info << ", (char**){ \"" << (ret == CURLE_OK ? *param.char_ptr : " <unchanged> ") << "\" }) = " << ret);
Dout(dc::curl, "curl_easy_getinfo(" << (AICURL*)curl << ", " << info << ", (char**){ \"" << (ret == CURLE_OK ? *param.char_ptr : " <unchanged> ") << "\" }) = " << ret);
break;
case CURLINFO_LONG:
Dout(dc::curl, "curl_easy_getinfo(" << curl << ", " << info << ", (long*){ " << (ret == CURLE_OK ? *param.long_ptr : 0L) << "L }) = " << ret);
Dout(dc::curl, "curl_easy_getinfo(" << (AICURL*)curl << ", " << info << ", (long*){ " << (ret == CURLE_OK ? *param.long_ptr : 0L) << "L }) = " << ret);
break;
case CURLINFO_DOUBLE:
Dout(dc::curl, "curl_easy_getinfo(" << curl << ", " << info << ", (double*){" << (ret == CURLE_OK ? *param.double_ptr : 0.) << "}) = " << ret);
Dout(dc::curl, "curl_easy_getinfo(" << (AICURL*)curl << ", " << info << ", (double*){" << (ret == CURLE_OK ? *param.double_ptr : 0.) << "}) = " << ret);
break;
case CURLINFO_SLIST:
Dout(dc::curl, "curl_easy_getinfo(" << curl << ", " << info << ", (curl_slist**){ " << (ret == CURLE_OK ? **param.curl_slist_ptr : unchanged_slist) << " }) = " << ret);
Dout(dc::curl, "curl_easy_getinfo(" << (AICURL*)curl << ", " << info << ", (curl_slist**){ " << (ret == CURLE_OK ? **param.curl_slist_ptr : unchanged_slist) << " }) = " << ret);
break;
}
}
@@ -624,7 +624,7 @@ void debug_curl_easy_reset(CURL* handle)
CURLcode debug_curl_easy_setopt(CURL* handle, CURLoption option, ...)
{
CURLcode ret;
CURLcode ret = CURLE_UNKNOWN_OPTION; // Suppress compiler warning.
va_list ap;
union param_type {
long along;
@@ -702,7 +702,11 @@ CURLcode debug_curl_easy_setopt(CURL* handle, CURLoption option, ...)
{
LibcwDoutStream << "NULL";
}
LibcwDoutStream << "](" << (is_postfield ? postfieldsize : size) << " bytes))";
LibcwDoutStream << "]";
if (str)
{
LibcwDoutStream << "(" << (is_postfield ? postfieldsize : size) << " bytes))";
}
}
else
{
@@ -755,7 +759,7 @@ char* debug_curl_easy_unescape(CURL* curl, char* url, int inlength, int* outleng
{
char* ret;
ret = curl_easy_unescape(curl, url, inlength, outlength);
Dout(dc::curl, "curl_easy_unescape(" << curl << ", \"" << url << "\", " << inlength << ", " << ((ret && outlength) ? *outlength : 1) << ") = \"" << ret << '"');
Dout(dc::curl, "curl_easy_unescape(" << (AICURL*)curl << ", \"" << url << "\", " << inlength << ", " << ((ret && outlength) ? *outlength : 1) << ") = \"" << ret << '"');
return ret;
}
@@ -837,7 +841,7 @@ CURLMcode debug_curl_multi_remove_handle(CURLM* multi_handle, CURL* easy_handle)
CURLMcode debug_curl_multi_setopt(CURLM* multi_handle, CURLMoption option, ...)
{
CURLMcode ret;
CURLMcode ret = CURLM_UNKNOWN_OPTION; // Suppress compiler warning.
va_list ap;
union param_type {
long along;

52
indra/llmessage/llavatarnamecache.cpp
View File

@@ -129,7 +129,7 @@ namespace LLAvatarNameCache
// Erase expired names from cache
void eraseUnrefreshed();
bool expirationFromCacheControl(LLSD headers, F64 *expires);
bool expirationFromCacheControl(AIHTTPReceivedHeaders const& headers, F64* expires);
}
/* Sample response:
@@ -171,7 +171,10 @@ namespace LLAvatarNameCache
</llsd>
*/
class LLAvatarNameResponder : public LLHTTPClient::Responder
class AIHTTPTimeoutPolicy;
extern AIHTTPTimeoutPolicy avatarNameResponder_timeout;
class LLAvatarNameResponder : public LLHTTPClient::ResponderWithResult
{
private:
// need to store agent ids that are part of this request in case of
@@ -179,24 +182,19 @@ private:
std::vector<LLUUID> mAgentIDs;
// Need the headers to look up Expires: and Retry-After:
LLSD mHeaders;
virtual bool needsHeaders(void) const { return true; }
public:
virtual AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const { return avatarNameResponder_timeout; }
LLAvatarNameResponder(const std::vector<LLUUID>& agent_ids)
: mAgentIDs(agent_ids),
mHeaders()
: mAgentIDs(agent_ids)
{ }
/*virtual*/ void completedHeader(U32 status, const std::string& reason,
const LLSD& headers)
{
mHeaders = headers;
}
/*virtual*/ void result(const LLSD& content)
{
// Pull expiration out of headers if available
F64 expires = LLAvatarNameCache::nameExpirationFromHeaders(mHeaders);
F64 expires = LLAvatarNameCache::nameExpirationFromHeaders(mReceivedHeaders);
F64 now = LLFrameTimer::getTotalSeconds();
LLSD agents = content["agents"];
@@ -788,43 +786,33 @@ void LLAvatarNameCache::insert(const LLUUID& agent_id, const LLAvatarName& av_na
sCache[agent_id] = av_name;
}
F64 LLAvatarNameCache::nameExpirationFromHeaders(LLSD headers)
F64 LLAvatarNameCache::nameExpirationFromHeaders(AIHTTPReceivedHeaders const& headers)
{
F64 expires = 0.0;
if (expirationFromCacheControl(headers, &expires))
{
return expires;
}
else
{
// With no expiration info, default to an hour
const F64 DEFAULT_EXPIRES = 60.0 * 60.0;
F64 now = LLFrameTimer::getTotalSeconds();
return now + DEFAULT_EXPIRES;
}
F64 expires;
expirationFromCacheControl(headers, &expires);
return expires;
}
bool LLAvatarNameCache::expirationFromCacheControl(LLSD headers, F64 *expires)
bool LLAvatarNameCache::expirationFromCacheControl(AIHTTPReceivedHeaders const& headers, F64* expires)
{
bool fromCacheControl = false;
S32 max_age = 3600; // With no expiration info, default to an hour.
F64 now = LLFrameTimer::getTotalSeconds();
// Allow the header to override the default
LLSD cache_control_header = headers["cache-control"];
if (cache_control_header.isDefined())
std::string cache_control;
if (headers.getFirstValue("cache-control", cache_control))
{
S32 max_age = 0;
std::string cache_control = cache_control_header.asString();
if (max_age_from_cache_control(cache_control, &max_age))
{
*expires = now + (F64)max_age;
fromCacheControl = true;
}
}
*expires = now + (F64)max_age;
LL_DEBUGS("AvNameCache")
<< ( fromCacheControl ? "expires based on cache control " : "default expiration " )
<< "in " << *expires - now << " seconds"
<< LL_ENDL;
return fromCacheControl;
}

4
indra/llmessage/llavatarnamecache.h
View File

@@ -32,8 +32,8 @@
#include <boost/signals2.hpp>
class LLSD;
class LLUUID;
class AIHTTPReceivedHeaders;
namespace LLAvatarNameCache
{
@@ -98,7 +98,7 @@ namespace LLAvatarNameCache
// Compute name expiration time from HTTP Cache-Control header,
// or return default value, in seconds from epoch.
F64 nameExpirationFromHeaders(LLSD headers);
F64 nameExpirationFromHeaders(AIHTTPReceivedHeaders const& headers);
void addUseDisplayNamesCallback(const use_display_name_signal_t::slot_type& cb);
}

32
indra/llmessage/llbuffer.cpp
View File

@@ -377,7 +377,7 @@ LLBufferArray::segment_iterator_t LLBufferArray::splitAfter(U8* address)
// We have the location and the segment.
U8* base = (*it).data();
S32 size = (*it).size();
if(address == (base + size))
if(address == (base + size - 1))
{
// No need to split, since this is the last byte of the
// segment. We do not want to have zero length segments, since
@@ -393,7 +393,14 @@ LLBufferArray::segment_iterator_t LLBufferArray::splitAfter(U8* address)
mSegments.insert(it, segment2);
return rv;
}
//mMutexp should be locked before calling this.
LLBufferArray::const_segment_iterator_t LLBufferArray::beginSegment() const
{
ASSERT_LLBUFFERARRAY_MUTEX_LOCKED
return mSegments.begin();
}
//mMutexp should be locked before calling this.
LLBufferArray::segment_iterator_t LLBufferArray::beginSegment()
{
@@ -401,6 +408,13 @@ LLBufferArray::segment_iterator_t LLBufferArray::beginSegment()
return mSegments.begin();
}
//mMutexp should be locked before calling this.
LLBufferArray::const_segment_iterator_t LLBufferArray::endSegment() const
{
ASSERT_LLBUFFERARRAY_MUTEX_LOCKED
return mSegments.end();
}
//mMutexp should be locked before calling this.
LLBufferArray::segment_iterator_t LLBufferArray::endSegment()
{
@@ -636,6 +650,20 @@ U8* LLBufferArray::readAfter(
return rv;
}
void LLBufferArray::writeChannelTo(std::ostream& ostr, S32 channel) const
{
LLMemType m1(LLMemType::MTYPE_IO_BUFFER);
LLMutexLock lock(mMutexp) ;
const_segment_iterator_t const end = mSegments.end();
for (const_segment_iterator_t it = mSegments.begin(); it != end; ++it)
{
if (it->isOnChannel(channel))
{
ostr.write((char*)it->data(), it->size());
}
}
}
U8* LLBufferArray::seek(
S32 channel,
U8* start,

10
indra/llmessage/llbuffer.h
View File

@@ -306,7 +306,7 @@ public:
typedef std::list<LLSegment> segment_list_t;
typedef segment_list_t::const_iterator const_segment_iterator_t;
typedef segment_list_t::iterator segment_iterator_t;
enum { npos = 0xffffffff };
static size_t const npos = (size_t)-1; // (U8*)npos is used as a magic address.
LLBufferArray();
~LLBufferArray();
@@ -495,6 +495,7 @@ public:
* @return Returns the segment if there is one.
*/
segment_iterator_t beginSegment();
const_segment_iterator_t beginSegment() const;
/**
* @brief Get the one-past-the-end segment in the buffer array
@@ -502,6 +503,7 @@ public:
* @return Returns the iterator for an invalid segment location.
*/
segment_iterator_t endSegment();
const_segment_iterator_t endSegment() const;
/**
* @brief Get the segment which holds the given address.
@@ -590,6 +592,12 @@ public:
void setThreaded(bool threaded);
//@}
/**
* @brief Read channel channel of LLBufferArray and write it to ostr.
* This is a Singularity extension.
*/
void writeChannelTo(std::ostream& ostr, S32 channel) const;
protected:
/**
* @brief Optimally put data in buffers, and reutrn segments.

14
indra/llmessage/llbufferstream.h
View File

@@ -118,6 +118,17 @@ protected:
//virtual streamsize xsputn(char* src, streamsize length);
//@}
public:
/*
* @brief Return number of bytes in input channel.
*/
S32 count_in(void) const { return mBuffer->count(mChannels.in()); }
/*
* @brief Return number of bytes in output channel.
*/
S32 count_out(void) const { return mBuffer->count(mChannels.out()); }
protected:
// This channels we are working on.
LLChannelDescriptors mChannels;
@@ -144,6 +155,9 @@ public:
LLBufferArray* buffer);
~LLBufferStream();
S32 count_in(void) const { return mStreamBuf.count_in(); }
S32 count_out(void) const { return mStreamBuf.count_out(); }
protected:
LLBufferStreamBuf mStreamBuf;
};

11
indra/llmessage/llcircuit.cpp
View File

@@ -1229,6 +1229,17 @@ void LLCircuit::getCircuitRange(
first = mCircuitData.upper_bound(key);
}
// <edit>
std::vector<LLCircuitData*> LLCircuit::getCircuitDataList()
{
std::vector<LLCircuitData*> list;
circuit_data_map::iterator end = mCircuitData.end();
for(circuit_data_map::iterator iter = mCircuitData.begin(); iter != end; ++iter)
list.push_back((*iter).second);
return list;
}
// </edit>
TPACKETID LLCircuitData::nextPacketOutID()
{
mPacketsOut++;

4
indra/llmessage/llcircuit.h
View File

@@ -328,6 +328,10 @@ public:
circuit_data_map::iterator& first,
circuit_data_map::iterator& end);
// <edit>
std::vector<LLCircuitData*> getCircuitDataList();
// </edit>
// Lists that optimize how many circuits we need to traverse a frame
// HACK - this should become protected eventually, but stupid !@$@# message system/circuit classes are jumbling things up.
circuit_data_map mUnackedCircuitMap; // Map of circuits with unacked data

154
indra/llmessage/llcurl.cpp
View File

@@ -1,154 +0,0 @@
/**
* @file llcurl.cpp
* @author Zero / Donovan
* @date 2006-10-15
* @brief Implementation of wrapper around libcurl.
*
* $LicenseInfo:firstyear=2006&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
//////////////////////////////////////////////////////////////////////////////
/*
The trick to getting curl to do keep-alives is to reuse the
same easy handle for the requests. It appears that curl
keeps a pool of connections alive for each easy handle, but
doesn't share them between easy handles. Therefore it is
important to keep a pool of easy handles and reuse them,
rather than create and destroy them with each request. This
code does this.
Furthermore, it would behoove us to keep track of which
hosts an easy handle was used for and pick an easy handle
that matches the next request. This code does not current
do this.
*/
//////////////////////////////////////////////////////////////////////////////
static const U32 EASY_HANDLE_POOL_SIZE = 5;
static const S32 MULTI_PERFORM_CALL_REPEAT = 5;
static const S32 CURL_REQUEST_TIMEOUT = 30; // seconds per operation
static const S32 MAX_ACTIVE_REQUEST_COUNT = 100;
//static
F32 LLCurl::sCurlRequestTimeOut = 120.f; //seconds
S32 LLCurl::sMaxHandles = 256; //max number of handles, (multi handles and easy handles combined).
//////////////////////////////////////////////////////////////////////////////
AIThreadSafeSimpleDC<LLCurl::Easy::Handles> LLCurl::Easy::sHandles;
//static
CURL* LLCurl::Easy::allocEasyHandle()
{
llassert(*AIAccess<LLCurlThread*>(LLCurl::getCurlThread())) ;
CURL* ret = NULL;
//*** Multi-threaded.
AIAccess<Handles> handles_w(sHandles);
if (handles_w->free.empty())
{
ret = LLCurl::newEasyHandle();
}
else
{
ret = *(handles_w->free.begin());
handles_w->free.erase(ret);
}
if (ret)
{
handles_w->active.insert(ret);
}
return ret;
}
//static
void LLCurl::Easy::releaseEasyHandle(CURL* handle)
{
DoutEntering(dc::curl, "LLCurl::Easy::releaseEasyHandle(" << (void*)handle << ")");
BACKTRACE;
static const S32 MAX_NUM_FREE_HANDLES = 32 ;
if (!handle)
{
return ; //handle allocation failed.
//llerrs << "handle cannot be NULL!" << llendl;
}
//*** Multi-Threaded (logout only?)
AIAccess<Handles> handles_w(sHandles);
if (handles_w->active.find(handle) != handles_w->active.end())
{
handles_w->active.erase(handle);
if (handles_w->free.size() < MAX_NUM_FREE_HANDLES)
{
curl_easy_reset(handle);
handles_w->free.insert(handle);
}
else
{
LLCurl::deleteEasyHandle(handle) ;
}
}
else
{
llerrs << "Invalid handle." << llendl;
}
}
LLCurl::Easy::~Easy()
{
AISTAccess<LLCurl::ResponderPtr> responder_w(mResponder);
if (*responder_w && LLCurl::getNotQuitting()) //aborted
{
std::string reason("Request timeout, aborted.") ;
(*responder_w)->completedRaw(408, //HTTP_REQUEST_TIME_OUT, timeout, abort
reason, mChannels, mOutput);
}
*responder_w = NULL;
}
LLCurl::Easy* LLCurlRequest::allocEasy()
{
if (!mActiveMulti ||
mActiveRequestCount >= MAX_ACTIVE_REQUEST_COUNT ||
mActiveMulti->mErrorCount > 0)
{
addMulti();
}
if(!mActiveMulti)
{
return NULL ;
}
//llassert_always(mActiveMulti);
++mActiveRequestCount;
LLCurl::Easy* easy = mActiveMulti->allocEasy();
return easy;
}

6
indra/llmessage/lldatapacker.h
View File

@@ -173,10 +173,12 @@ public:
void freeBuffer() { delete [] mBufferp; mBufferp = mCurBufferp = NULL; mBufferSize = 0; mWriteEnabled = FALSE; }
void assignBuffer(U8 *bufferp, S32 size)
{
if(mBufferp && mBufferp != bufferp)
//No no no no no! This breaks the paradigm of callers handling buffer allocation/deallocation
//Also, buffers can be on stack! Calling delete[] on such a buffer would be VERY bad.
/*if(mBufferp && mBufferp != bufferp)
{
freeBuffer() ;
}
}*/
mBufferp = bufferp;
mCurBufferp = bufferp;
mBufferSize = size;

10
indra/llmessage/llfiltersd2xmlrpc.cpp
View File

@@ -332,7 +332,7 @@ LLIOPipe::EStatus LLFilterSD2XMLRPCResponse::process_impl(
// we have everyting in the buffer, so turn the structure data rpc
// response into an xml rpc response.
LLBufferStream stream(channels, buffer.get());
stream << XML_HEADER << XMLRPC_METHOD_RESPONSE_HEADER;
stream << XML_HEADER << XMLRPC_METHOD_RESPONSE_HEADER << std::flush; // Flush, or buffer->count() returns too much!
LLSD sd;
LLSDSerialize::fromNotation(sd, stream, buffer->count(channels.in()));
@@ -484,7 +484,7 @@ LLIOPipe::EStatus LLFilterSD2XMLRPCRequest::process_impl(
break;
}
stream << XMLRPC_REQUEST_FOOTER;
stream << XMLRPC_REQUEST_FOOTER << std::flush;
return STATUS_DONE;
}
@@ -647,7 +647,7 @@ LLIOPipe::EStatus LLFilterXMLRPCResponse2LLSD::process_impl(
fault_string.assign(fault_str);
}
stream << "'" << LLSDNotationFormatter::escapeString(fault_string)
<< "'" <<LLSDRPC_FAULT_FOOTER;
<< "'" <<LLSDRPC_FAULT_FOOTER << std::flush;
}
else
{
@@ -658,7 +658,7 @@ LLIOPipe::EStatus LLFilterXMLRPCResponse2LLSD::process_impl(
{
stream_out(stream, param);
}
stream << LLSDRPC_RESPONSE_FOOTER;
stream << LLSDRPC_RESPONSE_FOOTER << std::flush;
}
PUMP_DEBUG;
XMLRPC_RequestFree(response, 1);
@@ -768,7 +768,7 @@ LLIOPipe::EStatus LLFilterXMLRPCRequest2LLSD::process_impl(
stream << "]";
}
}
stream << LLSDRPC_REQUEST_FOOTER;
stream << LLSDRPC_REQUEST_FOOTER << std::flush;
XMLRPC_RequestFree(request, 1);
delete[] buf;
PUMP_DEBUG;

959
indra/llmessage/llhttpclient.cpp
View File

File diff suppressed because it is too large Load Diff

454
indra/llmessage/llhttpclient.h
View File

@@ -32,85 +32,385 @@
*/
#include <string>
#include <curl/curl.h> // CURLcode
#include <boost/intrusive_ptr.hpp>
#include "llassettype.h"
#include "llcurl.h"
#include "lliopipe.h"
#include "llurlrequest.h"
extern const F32 HTTP_REQUEST_EXPIRY_SECS;
#include "llassettype.h"
#include "llhttpstatuscodes.h"
#include "aihttpheaders.h"
class LLUUID;
class LLPumpIO;
class LLSD;
class AIHTTPTimeoutPolicy;
class LLBufferArray;
class LLChannelDescriptors;
extern AIHTTPTimeoutPolicy responderIgnore_timeout;
typedef struct _xmlrpc_request* XMLRPC_REQUEST;
typedef struct _xmlrpc_value* XMLRPC_VALUE;
class LLHTTPClient
{
// Output parameter of AICurlPrivate::CurlEasyRequest::getResult.
// Used in XMLRPCResponder.
struct AITransferInfo {
AITransferInfo() : mSizeDownload(0.0), mTotalTime(0.0), mSpeedDownload(0.0) { }
F64 mSizeDownload;
F64 mTotalTime;
F64 mSpeedDownload;
};
// Events generated by AICurlPrivate::BufferedCurlEasyRequest
struct AIBufferedCurlEasyRequestEvents {
virtual void received_HTTP_header(void) = 0; // For example "HTTP/1.0 200 OK", the first header of a reply.
virtual void received_header(std::string const& key, std::string const& value) = 0; // Subsequent headers.
virtual void completed_headers(U32 status, std::string const& reason, AITransferInfo* info) = 0; // Transaction completed.
};
class LLHTTPClient {
public:
// class Responder moved to LLCurl
// For convenience
typedef LLCurl::Responder Responder;
typedef LLCurl::ResponderPtr ResponderPtr;
/** @name Responder base classes */
//@{
// The default actually already ignores responses.
class ResponderIgnore : public Responder { };
/**
* @class ResponderBase
* @brief Base class for all Responders.
*
* The life cycle of classes derived from this class is as follows:
* They are allocated with new on the line where get(), getByteRange() or post() is called,
* and the pointer to the allocated object is then put in a reference counting ResponderPtr.
* This ResponderPtr is passed to BufferedCurlEasyRequest::prepRequest which stores it in its
* member mResponder. Hence, the life time of a Responder is never longer than its
* associated BufferedCurlEasyRequest, however, if everything works correctly, then normally a
* responder is deleted in BufferedCurlEasyRequest::processOutput by setting
* mReponder to NULL.
*/
class ResponderBase : public AIBufferedCurlEasyRequestEvents {
public:
typedef boost::shared_ptr<LLBufferArray> buffer_ptr_t;
protected:
ResponderBase(void);
virtual ~ResponderBase();
// Read body from buffer and put it into content. If status indicates success, interpret it as LLSD, otherwise copy it as-is.
void decode_llsd_body(U32 status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer, LLSD& content);
// Read body from buffer and put it into content. Always copy it as-is.
void decode_raw_body(U32 status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer, std::string& content);
protected:
// Associated URL, used for debug output.
std::string mURL;
// Headers received from the server.
AIHTTPReceivedHeaders mReceivedHeaders;
// The curl result code.
CURLcode mCode;
// Set when the transaction finished (with or without errors).
bool mFinished;
public:
// Called to set the URL of the current request for this Responder,
// used only when printing debug output regarding activity of the Responder.
void setURL(std::string const& url);
// Accessors.
std::string const& getURL(void) const { return mURL; }
CURLcode result_code(void) const { return mCode; }
// Called by BufferedCurlEasyRequest::timed_out or BufferedCurlEasyRequest::processOutput.
virtual void finished(CURLcode code, U32 http_status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer) = 0;
// Return true if the curl thread is done with this transaction.
// If this returns true then it is guaranteed that none of the
// virtual functions will be called anymore: the curl thread
// will not access this object anymore.
// Note that normally you don't need to call this function.
bool is_finished(void) const { return mFinished; }
protected:
// AIBufferedCurlEasyRequestEvents
// These three events are only actually called for classes that implement a needsHeaders() that returns true.
// Called when the "HTTP/1.x <status> <reason>" header is received.
/*virtual*/ void received_HTTP_header(void)
{
// It's possible that this page was moved (302), so we already saw headers
// from the 302 page and are starting over on the new page now.
mReceivedHeaders.clear();
}
// Called for all remaining headers.
/*virtual*/ void received_header(std::string const& key, std::string const& value)
{
mReceivedHeaders.addHeader(key, value);
}
// Called when the whole transaction is completed (also the body was received), but before the body is processed.
/*virtual*/ void completed_headers(U32 status, std::string const& reason, AITransferInfo* info)
{
completedHeaders(status, reason, mReceivedHeaders);
}
public:
// Derived classes that implement completed_headers()/completedHeaders() should return true here.
virtual bool needsHeaders(void) const { return false; }
// A derived class should return true if curl should follow redirections.
// The default is not to follow redirections.
virtual bool followRedir(void) { return false; }
// Timeout policy to use.
virtual AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const = 0;
protected:
// Derived classes can override this to get the HTML headers that were received, when the message is completed.
// Only actually called for classes that implement a needsHeaders() that returns true.
virtual void completedHeaders(U32 status, std::string const& reason, AIHTTPReceivedHeaders const& headers)
{
// The default does nothing.
}
private:
// Used by ResponderPtr. Object is deleted when reference count reaches zero.
LLAtomicU32 mReferenceCount;
friend void intrusive_ptr_add_ref(ResponderBase* p); // Called by boost::intrusive_ptr when a new copy of a boost::intrusive_ptr<ResponderBase> is made.
friend void intrusive_ptr_release(ResponderBase* p); // Called by boost::intrusive_ptr when a boost::intrusive_ptr<ResponderBase> is destroyed.
// This function must delete the ResponderBase object when the reference count reaches zero.
};
// Responders derived from this base class should use HTTPClient::head or HTTPClient::getHeaderOnly.
// That will set the curl option CURLOPT_NOBODY so that only headers are received.
class ResponderHeadersOnly : public ResponderBase {
private:
/*virtual*/ bool needsHeaders(void) const { return true; }
protected:
// ResponderBase event
// The responder finished. Do not override this function in derived classes; override completedRaw instead.
/*virtual*/ void finished(CURLcode code, U32 http_status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer)
{
mCode = code;
// Allow classes derived from ResponderHeadersOnly to override completedHeaders.
completedHeaders(http_status, reason, mReceivedHeaders);
mFinished = true;
}
protected:
#ifdef SHOW_ASSERT
// Responders derived from this class must override completedHeaders.
// They may not attempt to override any of the virual functions defined by ResponderBase.
// Define those functions here with different parameters in order to cause a compile
// warning when a class accidently tries to override them.
enum YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS { };
virtual void completedRaw(YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS) { }
virtual void completed(YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS) { }
virtual void result(YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS) { }
virtual void errorWithContent(YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS) { }
virtual void error(YOU_MAY_ONLY_OVERRIDE_COMPLETED_HEADERS) { }
#endif
};
/**
* @class ResponderWithCompleted
* @brief Base class for Responders that implement completed, or completedRaw if the response is not LLSD.
*/
class ResponderWithCompleted : public ResponderBase {
protected:
// ResponderBase event
// The responder finished. Do not override this function in derived classes; override completedRaw instead.
/*virtual*/ void finished(CURLcode code, U32 http_status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer)
{
mCode = code;
// Allow classes derived from ResponderWithCompleted to override completedRaw
// (if not they should override completed or be derived from ResponderWithResult instead).
completedRaw(http_status, reason, channels, buffer);
mFinished = true;
}
protected:
// Events generated by this class.
// Derived classes can override this to get the raw data of the body of the HTML message that was received.
// The default is to interpret the content as LLSD and call completed().
virtual void completedRaw(U32 status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer);
// ... or, derived classes can override this to get LLSD content when the message is completed.
// The default aborts, as it should never be called (can't make it pure virtual though, so
// classes that override completedRaw don't need to implement this function, too).
virtual void completed(U32 status, std::string const& reason, LLSD const& content);
#ifdef SHOW_ASSERT
// Responders derived from this class must override either completedRaw or completed.
// They may not attempt to override any of the virual functions defined by ResponderBase.
// Define those functions here with different parameters in order to cause a compile
// warning when a class accidently tries to override them.
enum YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS { };
virtual void result(YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS) { }
virtual void errorWithContent(YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS) { }
virtual void error(YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS) { }
#endif
};
/**
* @class ResponderWithResult
* @brief Base class for reponders that expect LLSD in the body of the reply.
*
* Classes derived from ResponderWithResult must implement result, and either errorWithContent or error.
*/
class ResponderWithResult : public ResponderBase {
protected:
// The responder finished. Do not override this function in derived classes; use ResponderWithCompleted instead.
/*virtual*/ void finished(CURLcode code, U32 http_status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer);
protected:
// Events generated by this class.
// Derived classes must override this to receive the content of a body upon success.
virtual void result(LLSD const& content) = 0;
// Derived classes can override this to get informed when a bad HTML status code is received.
// The default calls error().
virtual void errorWithContent(U32 status, std::string const& reason, LLSD const& content);
// ... or, derived classes can override this to get informed when a bad HTML status code is received.
// The default prints the error to llinfos.
virtual void error(U32 status, std::string const& reason);
public:
// Called from LLSDMessage::ResponderAdapter::listener.
// LLSDMessage::ResponderAdapter is a hack, showing among others by fact that it needs these functions.
void pubErrorWithContent(CURLcode code, U32 status, std::string const& reason, LLSD const& content) { mCode = code; errorWithContent(status, reason, content); mFinished = true; }
void pubResult(LLSD const& content) { mCode = CURLE_OK; result(content); mFinished = true; }
#ifdef SHOW_ASSERT
// Responders derived from this class must override result, and either errorWithContent or error.
// They may not attempt to override any of the virual functions defined by ResponderWithCompleted.
// Define those functions here with different parameter in order to cause a compile
// warning when a class accidently tries to override them.
enum YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS { };
virtual void completedRaw(YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS) { }
virtual void completed(YOU_ARE_DERIVING_FROM_THE_WRONG_CLASS) { }
#endif
};
/**
* @class LegacyPolledResponder
* @brief As ResponderWithCompleted but caches the result for polling.
*
* This class allows old polling code to poll if the transaction finished
* by calling is_finished() (from the main the thread) and then access the
* results-- as opposed to immediately digesting the results when any of
* the virtual functions are called.
*/
class LegacyPolledResponder : public ResponderWithCompleted {
protected:
U32 mStatus;
std::string mReason;
protected:
// The responder finished. Do not override this function in derived classes.
/*virtual*/ void finished(CURLcode code, U32 http_status, std::string const& reason, LLChannelDescriptors const& channels, buffer_ptr_t const& buffer)
{
mStatus = http_status;
mReason = reason;
// Call base class implementation.
ResponderWithCompleted::finished(code, http_status, reason, channels, buffer);
}
public:
LegacyPolledResponder(void) : mStatus(HTTP_INTERNAL_ERROR) { }
// Accessors.
U32 http_status(void) const { return mStatus; }
std::string const& reason(void) const { return mReason; }
};
/**
* @class ResponderIgnoreBody
* @brief Base class for responders that ignore the result body.
*/
class ResponderIgnoreBody : public ResponderWithResult {
void result(LLSD const&) { }
};
/**
* @class ResponderIgnore
* @brief Responder that ignores the reply, if any, from the server.
*/
class ResponderIgnore : public ResponderIgnoreBody {
/*virtual*/ AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const { return responderIgnore_timeout;}
};
// A Responder is passed around as ResponderPtr, which causes it to automatically
// destruct when there are no pointers left pointing to it.
typedef boost::intrusive_ptr<ResponderBase> ResponderPtr;
//@}
/** @name non-blocking API */
//@{
static void head(
const std::string& url,
ResponderPtr,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void getByteRange(const std::string& url, S32 offset, S32 bytes, ResponderPtr, const LLSD& headers=LLSD(), const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void get(const std::string& url, ResponderPtr, const LLSD& headers = LLSD(), const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void get(const std::string& url, const LLSD& query, ResponderPtr, const LLSD& headers = LLSD(), const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void head(std::string const& url, ResponderHeadersOnly* responder, AIHTTPHeaders& headers);
static void head(std::string const& url, ResponderHeadersOnly* responder)
{ AIHTTPHeaders headers; head(url, responder, headers); }
static void put(
const std::string& url,
const LLSD& body,
ResponderPtr,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void getHeaderOnly(const std::string& url, ResponderPtr, const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void getHeaderOnly(const std::string& url, ResponderPtr, const LLSD& headers, const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void getByteRange(std::string const& url, S32 offset, S32 bytes, ResponderPtr responder, AIHTTPHeaders& headers);
static void getByteRange(std::string const& url, S32 offset, S32 bytes, ResponderPtr responder)
{ AIHTTPHeaders headers; getByteRange(url, offset, bytes, responder, headers); }
static void get(std::string const& url, ResponderPtr responder, AIHTTPHeaders& headers);
static void get(std::string const& url, ResponderPtr responder)
{ AIHTTPHeaders headers; get(url, responder, headers); }
static void get(std::string const& url, LLSD const& query, ResponderPtr responder, AIHTTPHeaders& headers);
static void get(std::string const& url, LLSD const& query, ResponderPtr responder)
{ AIHTTPHeaders headers; get(url, query, responder, headers); }
static void put(std::string const& url, LLSD const& body, ResponderPtr responder, AIHTTPHeaders& headers);
static void put(std::string const& url, LLSD const& body, ResponderPtr responder)
{ AIHTTPHeaders headers; put(url, body, responder, headers); }
static void getHeaderOnly(std::string const& url, ResponderHeadersOnly* responder, AIHTTPHeaders& headers);
static void getHeaderOnly(std::string const& url, ResponderHeadersOnly* responder)
{ AIHTTPHeaders headers; getHeaderOnly(url, responder, headers); }
static void post(std::string const& url, LLSD const& body, ResponderPtr responder, AIHTTPHeaders& headers);
static void post(std::string const& url, LLSD const& body, ResponderPtr responder)
{ AIHTTPHeaders headers; post(url, body, responder, headers); }
/** Takes ownership of request and deletes it when sent */
static void postXMLRPC(std::string const& url, XMLRPC_REQUEST request, ResponderPtr responder, AIHTTPHeaders& headers);
static void postXMLRPC(std::string const& url, XMLRPC_REQUEST request, ResponderPtr responder)
{ AIHTTPHeaders headers; postXMLRPC(url, request, responder, headers); }
static void postXMLRPC(std::string const& url, char const* method, XMLRPC_VALUE value, ResponderPtr responder, AIHTTPHeaders& headers);
static void postXMLRPC(std::string const& url, char const* method, XMLRPC_VALUE value, ResponderPtr responder)
{ AIHTTPHeaders headers; postXMLRPC(url, method, value, responder, headers); }
static void post(
const std::string& url,
const LLSD& body,
ResponderPtr,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
/** Takes ownership of data and deletes it when sent */
static void postRaw(
const std::string& url,
const U8* data,
S32 size,
ResponderPtr responder,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void postFile(
const std::string& url,
const std::string& filename,
ResponderPtr,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void postFile(
const std::string& url,
const LLUUID& uuid,
LLAssetType::EType asset_type,
ResponderPtr responder,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void postRaw(std::string const& url, const char* data, S32 size, ResponderPtr responder, AIHTTPHeaders& headers);
static void postRaw(std::string const& url, const char* data, S32 size, ResponderPtr responder)
{ AIHTTPHeaders headers; postRaw(url, data, size, responder, headers); }
static void postFile(std::string const& url, std::string const& filename, ResponderPtr responder, AIHTTPHeaders& headers);
static void postFile(std::string const& url, std::string const& filename, ResponderPtr responder)
{ AIHTTPHeaders headers; postFile(url, filename, responder, headers); }
static void postFile(std::string const& url, const LLUUID& uuid, LLAssetType::EType asset_type, ResponderPtr responder, AIHTTPHeaders& headers);
static void postFile(std::string const& url, const LLUUID& uuid, LLAssetType::EType asset_type, ResponderPtr responder)
{ AIHTTPHeaders headers; postFile(url, uuid, asset_type, responder, headers); }
static void del(std::string const& url, ResponderPtr responder, AIHTTPHeaders& headers);
static void del(std::string const& url, ResponderPtr responder)
{ AIHTTPHeaders headers; del(url, responder, headers); }
static void del(
const std::string& url,
ResponderPtr responder,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
///< sends a DELETE method, but we can't call it delete in c++
/**
@@ -122,22 +422,28 @@ public:
* @param headers A map of key:value headers to pass to the request
* @param timeout The number of seconds to give the server to respond.
*/
static void move(
const std::string& url,
const std::string& destination,
ResponderPtr responder,
const LLSD& headers = LLSD(),
const F32 timeout=HTTP_REQUEST_EXPIRY_SECS);
static void move(std::string const& url, std::string const& destination, ResponderPtr responder, AIHTTPHeaders& headers);
static void move(std::string const& url, std::string const& destination, ResponderPtr responder)
{ AIHTTPHeaders headers; move(url, destination, responder, headers); }
//@}
/**
* @brief Blocking HTTP get that returns an LLSD map of status and body.
* @brief Blocking HTTP GET that returns an LLSD map of status and body.
*
* @param url the complete serialized (and escaped) url to get
* @return An LLSD of { 'status':status, 'body':payload }
*/
static LLSD blockingGet(const std::string& url);
static LLSD blockingGet(std::string const& url);
/**
* @brief Blocking HTTP GET that returns the raw body.
*
* @param url the complete serialized (and escaped) url to get
* @param result the target string to write the body to
* @return HTTP status
*/
static U32 blockingGetRaw(const std::string& url, std::string& result);
/**
* @brief Blocking HTTP POST that returns an LLSD map of status and body.
@@ -146,15 +452,7 @@ public:
* @param body the LLSD post body
* @return An LLSD of { 'status':status (an int), 'body':payload (an LLSD) }
*/
static LLSD blockingPost(const std::string& url, const LLSD& body);
static void setPump(LLPumpIO& pump);
///< must be called before any of the above calls are made
static bool hasPump();
///< for testing
static LLPumpIO &getPump();
///< Hippo special
static LLSD blockingPost(std::string const& url, LLSD const& body);
};
#endif // LL_LLHTTPCLIENT_H

55
indra/llmessage/llhttpclientadapter.cpp
View File

@@ -1,55 +0,0 @@
/**
* @file llhttpclientadapter.cpp
* @brief
*
* $LicenseInfo:firstyear=2009&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#include "llhttpclientadapter.h"
#include "llhttpclient.h"
LLHTTPClientAdapter::~LLHTTPClientAdapter()
{
}
void LLHTTPClientAdapter::get(const std::string& url, LLCurl::ResponderPtr responder)
{
LLSD empty_pragma_header;
// Pragma is required to stop curl adding "no-cache"
// Space is required to stop llurlrequest from turnning off proxying
empty_pragma_header["Pragma"] = " ";
LLHTTPClient::get(url, responder, empty_pragma_header);
}
void LLHTTPClientAdapter::get(const std::string& url, LLCurl::ResponderPtr responder, const LLSD& headers)
{
LLSD empty_pragma_header = headers;
// as above
empty_pragma_header["Pragma"] = " ";
LLHTTPClient::get(url, responder, empty_pragma_header);
}
void LLHTTPClientAdapter::put(const std::string& url, const LLSD& body, LLCurl::ResponderPtr responder)
{
LLHTTPClient::put(url, body, responder);
}

43
indra/llmessage/llhttpclientadapter.h
View File

@@ -1,43 +0,0 @@
/**
* @file llhttpclientadepter.h
* @brief
*
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#ifndef LL_HTTPCLIENTADAPTER_H
#define LL_HTTPCLIENTADAPTER_H
#include "llhttpclientinterface.h"
#include "llsingleton.h" // LLSingleton<>
class LLHTTPClientAdapter : public LLHTTPClientInterface, public LLSingleton<LLHTTPClientAdapter>
{
public:
virtual ~LLHTTPClientAdapter();
virtual void get(const std::string& url, LLCurl::ResponderPtr responder);
virtual void get(const std::string& url, LLCurl::ResponderPtr responder, const LLSD& headers);
virtual void put(const std::string& url, const LLSD& body, LLCurl::ResponderPtr responder);
};
#endif

45
indra/llmessage/llhttpclientinterface.h
View File

@@ -1,45 +0,0 @@
/**
* @file llhttpclientinterface.h
* @brief
*
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#ifndef LL_LLHTTPCLIENTINTERFACE_H
#define LL_LLHTTPCLIENTINTERFACE_H
#include "linden_common.h"
#include "llcurl.h"
#include <string>
class LLHTTPClientInterface
{
public:
virtual ~LLHTTPClientInterface() {}
virtual void get(const std::string& url, LLCurl::ResponderPtr responder) = 0;
virtual void get(const std::string& url, LLCurl::ResponderPtr responder, const LLSD& headers) = 0;
virtual void put(const std::string& url, const LLSD& body, LLCurl::ResponderPtr responder) = 0;
};
#endif // LL_LLHTTPCLIENTINTERFACE_H

9
indra/llmessage/lliohttpserver.cpp
View File

@@ -272,6 +272,7 @@ LLIOPipe::EStatus LLHTTPPipe::process_impl(
context[CONTEXT_RESPONSE][CONTEXT_HEADERS] = headers;
LLBufferStream ostr(channels, buffer.get());
LLSDSerialize::toXML(mGoodResult, ostr);
ostr << std::flush;
return STATUS_DONE;
}
@@ -284,7 +285,7 @@ LLIOPipe::EStatus LLHTTPPipe::process_impl(
context[CONTEXT_RESPONSE]["statusCode"] = mStatusCode;
context[CONTEXT_RESPONSE]["statusMessage"] = mStatusMessage;
LLBufferStream ostr(channels, buffer.get());
ostr << mStatusMessage;
ostr << mStatusMessage << std::flush;
return STATUS_DONE;
}
@@ -293,7 +294,7 @@ LLIOPipe::EStatus LLHTTPPipe::process_impl(
context[CONTEXT_RESPONSE][CONTEXT_HEADERS] = mHeaders;
context[CONTEXT_RESPONSE]["statusCode"] = mStatusCode;
LLBufferStream ostr(channels, buffer.get());
ostr << mStatusMessage;
ostr << mStatusMessage << std::flush;
return STATUS_DONE;
}
@@ -633,7 +634,7 @@ void LLHTTPResponder::markBad(
LLBufferStream out(channels, buffer.get());
out << HTTP_VERSION_STR << " 400 Bad Request\r\n\r\n<html>\n"
<< "<title>Bad Request</title>\n<body>\nBad Request.\n"
<< "</body>\n</html>\n";
<< "</body>\n</html>\n" << std::flush;
}
static LLFastTimer::DeclareTimer FTM_PROCESS_HTTP_RESPONDER("HTTP Responder");
@@ -926,7 +927,7 @@ LLIOPipe::EStatus LLHTTPResponder::process_impl(
mState = STATE_SHORT_CIRCUIT;
str << HTTP_VERSION_STR << " 404 Not Found\r\n\r\n<html>\n"
<< "<title>Not Found</title>\n<body>\nNode '" << mAbsPathAndQuery
<< "' not found.\n</body>\n</html>\n";
<< "' not found.\n</body>\n</html>\n" << std::flush;
}
}

54
indra/llmessage/llmessagelog.cpp Normal file
View File

@@ -0,0 +1,54 @@
// <edit>
#include "llmessagelog.h"
LLMessageLogEntry::LLMessageLogEntry(EType type, LLHost from_host, LLHost to_host, U8* data, S32 data_size)
: mType(type),
mFromHost(from_host),
mToHost(to_host),
mDataSize(data_size)
{
if(data)
{
mData.resize(data_size);
memcpy(&(mData[0]), data, data_size);
}
}
LLMessageLogEntry::LLMessageLogEntry(EType type, LLHost from_host, LLHost to_host, std::vector<U8> data, S32 data_size)
: mType(type),
mFromHost(from_host),
mToHost(to_host),
mDataSize(data_size),
mData(data)
{
}
LLMessageLogEntry::~LLMessageLogEntry()
{
}
U32 LLMessageLog::sMaxSize = 4096; // testzone fixme todo boom
std::deque<LLMessageLogEntry> LLMessageLog::sDeque;
void (*(LLMessageLog::sCallback))(LLMessageLogEntry);
void LLMessageLog::setMaxSize(U32 size)
{
sMaxSize = size;
while(sDeque.size() > sMaxSize)
sDeque.pop_front();
}
void LLMessageLog::setCallback(void (*callback)(LLMessageLogEntry))
{
sCallback = callback;
}
void LLMessageLog::log(LLHost from_host, LLHost to_host, U8* data, S32 data_size)
{
LLMessageLogEntry entry = LLMessageLogEntry(LLMessageLogEntry::TEMPLATE, from_host, to_host, data, data_size);
if(!entry.mDataSize || !entry.mData.size()) return;
if(sCallback) sCallback(entry);
if(!sMaxSize) return;
sDeque.push_back(entry);
if(sDeque.size() > sMaxSize)
sDeque.pop_front();
}
std::deque<LLMessageLogEntry> LLMessageLog::getDeque()
{
return sDeque;
}
// </edit>

41
indra/llmessage/llmessagelog.h Normal file
View File

@@ -0,0 +1,41 @@
// <edit>
#ifndef LL_LLMESSAGELOG_H
#define LL_LLMESSAGELOG_H
#include "stdtypes.h"
#include "llhost.h"
#include <queue>
#include <string.h>
class LLMessageSystem;
class LLMessageLogEntry
{
public:
enum EType
{
TEMPLATE,
HTTP_REQUEST,
HTTP_RESPONSE
};
LLMessageLogEntry(EType type, LLHost from_host, LLHost to_host, U8* data, S32 data_size);
LLMessageLogEntry(EType type, LLHost from_host, LLHost to_host, std::vector<U8> data, S32 data_size);
~LLMessageLogEntry();
EType mType;
LLHost mFromHost;
LLHost mToHost;
S32 mDataSize;
std::vector<U8> mData;
};
class LLMessageLog
{
public:
static void setMaxSize(U32 size);
static void setCallback(void (*callback)(LLMessageLogEntry));
static void log(LLHost from_host, LLHost to_host, U8* data, S32 data_size);
static std::deque<LLMessageLogEntry> getDeque();
private:
static U32 sMaxSize;
static void (*sCallback)(LLMessageLogEntry);
static std::deque<LLMessageLogEntry> sDeque;
};
#endif
// </edit>

10
indra/llmessage/llpacketring.cpp
View File

@@ -44,6 +44,10 @@
#include "timing.h"
#include "u64.h"
//<edit>
#include "llmessagelog.h"
//</edit>
///////////////////////////////////////////////////////////
LLPacketRing::LLPacketRing () :
mUseInThrottle(FALSE),
@@ -272,7 +276,11 @@ S32 LLPacketRing::receivePacket (S32 socket, char *datap)
}
BOOL LLPacketRing::sendPacket(int h_socket, char * send_buffer, S32 buf_size, LLHost host)
{
{
//<edit>
LLMessageLog::log(LLHost(16777343, gMessageSystem->getListenPort()), host, (U8*)send_buffer, buf_size);
//</edit>
BOOL status = TRUE;
if (!mUseOutThrottle)
{

6
indra/llmessage/llpartdata.cpp
View File

@@ -238,14 +238,14 @@ BOOL LLPartSysData::unpack(LLDataPacker &dp)
std::ostream& operator<<(std::ostream& s, const LLPartSysData &data)
{
s << "Flags: " << std::hex << data.mFlags;
s << " Pattern: " << std::hex << (U32) data.mPattern << "\n";
s << "Flags: " << std::hex << data.mFlags << std::dec;
s << " Pattern: " << std::hex << (U32) data.mPattern << std::dec << "\n";
s << "Age: [" << data.mStartAge << ", " << data.mMaxAge << "]\n";
s << "Angle: [" << data.mInnerAngle << ", " << data.mOuterAngle << "]\n";
s << "Burst Rate: " << data.mBurstRate << "\n";
s << "Burst Radius: " << data.mBurstRadius << "\n";
s << "Burst Speed: [" << data.mBurstSpeedMin << ", " << data.mBurstSpeedMax << "]\n";
s << "Burst Part Count: " << std::hex << (U32) data.mBurstPartCount << "\n";
s << "Burst Part Count: " << std::hex << (U32) data.mBurstPartCount << std::dec << "\n";
s << "Angular Velocity: " << data.mAngularVelocity << "\n";
s << "Accel: " << data.mPartAccel;
return s;

153
indra/llmessage/llregionpresenceverifier.cpp
View File

@@ -1,153 +0,0 @@
/**
* @file llregionpresenceverifier.cpp
* @brief
*
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#include "linden_common.h"
#include "llregionpresenceverifier.h"
#include "llhttpclientinterface.h"
#include <sstream>
#include "net.h"
#include "message.h"
//namespace boost
//{
void intrusive_ptr_add_ref(LLRegionPresenceVerifier::Response* p)
{
++p->mReferenceCount;
}
void intrusive_ptr_release(LLRegionPresenceVerifier::Response* p)
{
if(p && 0 == --p->mReferenceCount)
{
delete p;
}
}
//};
LLRegionPresenceVerifier::Response::~Response()
{
}
LLRegionPresenceVerifier::RegionResponder::RegionResponder(const std::string&
uri,
ResponsePtr data,
S32 retry_count) :
mUri(uri),
mSharedData(data),
mRetryCount(retry_count)
{
}
//virtual
LLRegionPresenceVerifier::RegionResponder::~RegionResponder()
{
}
void LLRegionPresenceVerifier::RegionResponder::result(const LLSD& content)
{
std::string host = content["private_host"].asString();
U32 port = content["private_port"].asInteger();
LLHost destination(host, port);
LLUUID id = content["region_id"];
lldebugs << "Verifying " << destination.getString() << " is region " << id << llendl;
std::stringstream uri;
uri << "http://" << destination.getString() << "/state/basic/";
mSharedData->getHttpClient().get(
uri.str(),
new VerifiedDestinationResponder(mUri, mSharedData, content, mRetryCount));
}
void LLRegionPresenceVerifier::RegionResponder::error(U32 status,
const std::string& reason)
{
// TODO: babbage: distinguish between region presence service and
// region verification errors?
mSharedData->onRegionVerificationFailed();
}
LLRegionPresenceVerifier::VerifiedDestinationResponder::VerifiedDestinationResponder(const std::string& uri, ResponsePtr data, const LLSD& content,
S32 retry_count):
mUri(uri),
mSharedData(data),
mContent(content),
mRetryCount(retry_count)
{
}
//virtual
LLRegionPresenceVerifier::VerifiedDestinationResponder::~VerifiedDestinationResponder()
{
}
void LLRegionPresenceVerifier::VerifiedDestinationResponder::result(const LLSD& content)
{
LLUUID actual_region_id = content["region_id"];
LLUUID expected_region_id = mContent["region_id"];
lldebugs << "Actual region: " << content << llendl;
lldebugs << "Expected region: " << mContent << llendl;
if (mSharedData->checkValidity(content) &&
(actual_region_id == expected_region_id))
{
mSharedData->onRegionVerified(mContent);
}
else if (mRetryCount > 0)
{
retry();
}
else
{
llwarns << "Simulator verification failed. Region: " << mUri << llendl;
mSharedData->onRegionVerificationFailed();
}
}
void LLRegionPresenceVerifier::VerifiedDestinationResponder::retry()
{
LLSD headers;
headers["Cache-Control"] = "no-cache, max-age=0";
llinfos << "Requesting region information, get uncached for region "
<< mUri << llendl;
--mRetryCount;
mSharedData->getHttpClient().get(mUri, new RegionResponder(mUri, mSharedData, mRetryCount), headers);
}
void LLRegionPresenceVerifier::VerifiedDestinationResponder::error(U32 status, const std::string& reason)
{
if(mRetryCount > 0)
{
retry();
}
else
{
llwarns << "Failed to contact simulator for verification. Region: " << mUri << llendl;
mSharedData->onRegionVerificationFailed();
}
}

98
indra/llmessage/llregionpresenceverifier.h
View File

@@ -1,98 +0,0 @@
/**
* @file llregionpresenceverifier.cpp
* @brief
*
* $LicenseInfo:firstyear=2008&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
/* Macro Definitions */
#ifndef LL_LLREGIONPRESENCEVERIFIER_H
#define LL_LLREGIONPRESENCEVERIFIER_H
#include "llhttpclient.h"
#include <string>
#include "llsd.h"
#include <boost/intrusive_ptr.hpp>
class LLHTTPClientInterface;
class LLRegionPresenceVerifier
{
public:
class Response
{
public:
virtual ~Response() = 0;
virtual bool checkValidity(const LLSD& content) const = 0;
virtual void onRegionVerified(const LLSD& region_details) = 0;
virtual void onRegionVerificationFailed() = 0;
virtual LLHTTPClientInterface& getHttpClient() = 0;
public: /* but not really -- don't touch this */
U32 mReferenceCount;
};
typedef boost::intrusive_ptr<Response> ResponsePtr;
class RegionResponder : public LLHTTPClient::Responder
{
public:
RegionResponder(const std::string& uri, ResponsePtr data,
S32 retry_count);
virtual ~RegionResponder();
virtual void result(const LLSD& content);
virtual void error(U32 status, const std::string& reason);
private:
ResponsePtr mSharedData;
std::string mUri;
S32 mRetryCount;
};
class VerifiedDestinationResponder : public LLHTTPClient::Responder
{
public:
VerifiedDestinationResponder(const std::string& uri, ResponsePtr data,
const LLSD& content, S32 retry_count);
virtual ~VerifiedDestinationResponder();
virtual void result(const LLSD& content);
virtual void error(U32 status, const std::string& reason);
private:
void retry();
ResponsePtr mSharedData;
LLSD mContent;
std::string mUri;
S32 mRetryCount;
};
};
//namespace boost
//{
void intrusive_ptr_add_ref(LLRegionPresenceVerifier::Response* p);
void intrusive_ptr_release(LLRegionPresenceVerifier::Response* p);
//};
#endif //LL_LLREGIONPRESENCEVERIFIER_H

50
indra/llmessage/llsdmessage.cpp
View File

@@ -45,6 +45,7 @@
#include "llhost.h"
#include "message.h"
#include "llsdutil.h"
#include "aihttptimeoutpolicy.h"
// Declare a static LLSDMessage instance to ensure that we have a listener as
// soon as someone tries to post on our canonical LLEventPump name.
@@ -62,13 +63,15 @@ LLSDMessage::LLSDMessage():
bool LLSDMessage::httpListener(const LLSD& request)
{
llassert(false); // This function is never called. --Aleric
// Extract what we want from the request object. We do it all up front
// partly to document what we expect.
LLSD::String url(request["url"]);
LLSD payload(request["payload"]);
LLSD::String reply(request["reply"]);
LLSD::String error(request["error"]);
LLSD::Real timeout(request["timeout"]);
LLSD::String timeoutpolicy(request["timeoutpolicy"]);
// If the LLSD doesn't even have a "url" key, we doubt it was intended for
// this listener.
if (url.empty())
@@ -77,21 +80,25 @@ bool LLSDMessage::httpListener(const LLSD& request)
out << "request event without 'url' key to '" << mEventPump.getName() << "'";
throw ArgError(out.str());
}
// Establish default timeout. This test relies on LLSD::asReal() returning
// exactly 0.0 for an undef value.
if (! timeout)
{
timeout = HTTP_REQUEST_EXPIRY_SECS;
}
LLHTTPClient::post(url, payload,
new LLSDMessage::EventResponder(LLEventPumps::instance(),
request,
url, "POST", reply, error),
LLSD(), // headers
(F32)timeout);
LLSDMessage::EventResponder* responder =
new LLSDMessage::EventResponder(LLEventPumps::instance(), request, url, "POST", reply, error);
responder->setTimeoutPolicy(timeoutpolicy);
LLHTTPClient::post(url, payload, responder);
return false;
}
LLSDMessage::EventResponder::EventResponder(LLEventPumps& pumps, LLSD const& request, std::string const& target,
std::string const& message, std::string const& replyPump, std::string const& errorPump) :
mPumps(pumps), mReqID(request), mTarget(target), mMessage(message), mReplyPump(replyPump), mErrorPump(errorPump),
mHTTPTimeoutPolicy(AIHTTPTimeoutPolicy::getTimeoutPolicyByName(std::string()))
{
}
void LLSDMessage::EventResponder::setTimeoutPolicy(std::string const& name)
{
mHTTPTimeoutPolicy = AIHTTPTimeoutPolicy::getTimeoutPolicyByName(name);
}
void LLSDMessage::EventResponder::result(const LLSD& data)
{
// If our caller passed an empty replyPump name, they're not
@@ -121,6 +128,7 @@ void LLSDMessage::EventResponder::errorWithContent(U32 status, const std::string
LLSD info(mReqID.makeResponse());
info["target"] = mTarget;
info["message"] = mMessage;
info["code"] = mCode;
info["status"] = LLSD::Integer(status);
info["reason"] = reason;
info["content"] = content;
@@ -137,7 +145,7 @@ void LLSDMessage::EventResponder::errorWithContent(U32 status, const std::string
}
}
LLSDMessage::ResponderAdapter::ResponderAdapter(LLHTTPClient::ResponderPtr responder,
LLSDMessage::ResponderAdapter::ResponderAdapter(LLHTTPClient::ResponderWithResult* responder,
const std::string& name):
mResponder(responder),
mReplyPump(name + ".reply", true), // tweak name for uniqueness
@@ -147,15 +155,25 @@ LLSDMessage::ResponderAdapter::ResponderAdapter(LLHTTPClient::ResponderPtr respo
mErrorPump.listen("self", boost::bind(&ResponderAdapter::listener, this, _1, false));
}
std::string LLSDMessage::ResponderAdapter::getTimeoutPolicyName(void) const
{
return mResponder->getHTTPTimeoutPolicy().name();
}
bool LLSDMessage::ResponderAdapter::listener(const LLSD& payload, bool success)
{
LLHTTPClient::ResponderWithResult* responder = dynamic_cast<LLHTTPClient::ResponderWithResult*>(mResponder.get());
// If this assertion fails then ResponderAdapter has been used for a ResponderWithCompleted derived class,
// which is not allowed because ResponderAdapter can only work for classes derived from Responder that
// implement result() and errorWithContent (or just error).
llassert_always(responder);
if (success)
{
mResponder->pubResult(payload);
responder->pubResult(payload);
}
else
{
mResponder->pubErrorWithContent(payload["status"].asInteger(), payload["reason"], payload["content"]);
responder->pubErrorWithContent((CURLcode)payload["code"].asInteger(), payload["status"].asInteger(), payload["reason"], payload["content"]);
}
/*---------------- MUST BE LAST STATEMENT BEFORE RETURN ----------------*/

38
indra/llmessage/llsdmessage.h
View File

@@ -37,6 +37,8 @@
#include <stdexcept>
class LLSD;
class AIHTTPTimeoutPolicy;
extern AIHTTPTimeoutPolicy eventResponder_timeout;
/**
* Class managing the messaging API described in
@@ -64,37 +66,47 @@ public:
* must be visible to the reply/error methods can conveniently be stored
* on that class itself, if it's not already.
*
* The LLHTTPClient::Responder idiom requires a separate instance of a
* The LLHTTPClient::ResponderBase idiom requires a separate instance of a
* separate class so that it can dispatch to the code of interest by
* calling canonical virtual methods. Interesting state must be copied
* into that new object.
*
* With some trepidation, because existing response code is packaged in
* LLHTTPClient::Responder subclasses, we provide this adapter class
* LLHTTPClient::ResponderWithResult subclasses, we provide this adapter class
* <i>for transitional purposes only.</i> Instantiate a new heap
* ResponderAdapter with your new LLHTTPClient::ResponderPtr. Pass
* ResponderAdapter::getReplyName() and/or getErrorName() in your
* LLSDMessage (or LLViewerRegion::getCapAPI()) request event. The
* ResponderAdapter will call the appropriate Responder method, then
* @c delete itself.
*
* Singularity note: I think this class/API is a bad idea that makes things
* more complex, a lot slower and less OO. The idea to get methods called
* on the same class that does the request is a nice idea, but should
* be implemented through boost::bind and NOT use LLSD. Avoid.
* Also note that this only works for ResponderWithResult derived classes,
* not for responders derived from ResponderWithCompleted.
* --Aleric
*/
class ResponderAdapter
{
public:
/**
* Bind the new LLHTTPClient::Responder subclass instance.
* Bind the new LLHTTPClient::ResponderWithResult subclass instance.
*
* Passing the constructor a name other than the default is only
* interesting if you suspect some usage will lead to an exception or
* log message.
*/
ResponderAdapter(LLHTTPClient::ResponderPtr responder,
ResponderAdapter(LLHTTPClient::ResponderWithResult* responder,
const std::string& name="ResponderAdapter");
/// EventPump name on which LLSDMessage should post reply event
std::string getReplyName() const { return mReplyPump.getName(); }
/// EventPump name on which LLSDMessage should post error event
std::string getErrorName() const { return mErrorPump.getName(); }
/// Name of timeout policy to use.
std::string getTimeoutPolicyName() const;
private:
// We have two different LLEventStreams, though we route them both to
@@ -121,11 +133,13 @@ private:
friend class LLCapabilityListener;
/// Responder used for internal purposes by LLSDMessage and
/// LLCapabilityListener. Others should use higher-level APIs.
class EventResponder: public LLHTTPClient::Responder
class EventResponder: public LLHTTPClient::ResponderWithResult
{
public:
virtual AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const { return *mHTTPTimeoutPolicy; }
/**
* LLHTTPClient::Responder that dispatches via named LLEventPump instances.
* LLHTTPClient::ResponderWithResult that dispatches via named LLEventPump instances.
* We bind LLEventPumps, even though it's an LLSingleton, for testability.
* We bind the string names of the desired LLEventPump instances rather
* than actually obtain()ing them so we only obtain() the one we're going
@@ -140,14 +154,9 @@ private:
EventResponder(LLEventPumps& pumps,
const LLSD& request,
const std::string& target, const std::string& message,
const std::string& replyPump, const std::string& errorPump):
mPumps(pumps),
mReqID(request),
mTarget(target),
mMessage(message),
mReplyPump(replyPump),
mErrorPump(errorPump)
{}
const std::string& replyPump, const std::string& errorPump);
void setTimeoutPolicy(std::string const& name);
virtual void result(const LLSD& data);
virtual void errorWithContent(U32 status, const std::string& reason, const LLSD& content);
@@ -156,6 +165,7 @@ private:
LLEventPumps& mPumps;
LLReqID mReqID;
const std::string mTarget, mMessage, mReplyPump, mErrorPump;
AIHTTPTimeoutPolicy const* mHTTPTimeoutPolicy;
};
private:

111
indra/llmessage/llservice.cpp
View File

@@ -1,111 +0,0 @@
/**
* @file llservice.cpp
* @author Phoenix
* @date 2005-04-20
*
* $LicenseInfo:firstyear=2005&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#include "linden_common.h"
#include "llservice.h"
LLService::creators_t LLService::sCreatorFunctors;
LLService::LLService()
{
}
LLService::~LLService()
{
}
// static
bool LLService::registerCreator(const std::string& name, creator_t fn)
{
llinfos << "LLService::registerCreator(" << name << ")" << llendl;
if(name.empty())
{
return false;
}
creators_t::value_type vt(name, fn);
std::pair<creators_t::iterator, bool> rv = sCreatorFunctors.insert(vt);
return rv.second;
// alternately...
//std::string name_str(name);
//sCreatorFunctors[name_str] = fn;
}
// static
LLIOPipe* LLService::activate(
const std::string& name,
LLPumpIO::chain_t& chain,
LLSD context)
{
if(name.empty())
{
llinfos << "LLService::activate - no service specified." << llendl;
return NULL;
}
creators_t::iterator it = sCreatorFunctors.find(name);
LLIOPipe* rv = NULL;
if(it != sCreatorFunctors.end())
{
if((*it).second->build(chain, context))
{
rv = chain[0].get();
}
else
{
// empty out the chain, because failed service creation
// should just discard this stuff.
llwarns << "LLService::activate - unable to build chain: " << name
<< llendl;
chain.clear();
}
}
else
{
llwarns << "LLService::activate - unable find factory: " << name
<< llendl;
}
return rv;
}
// static
bool LLService::discard(const std::string& name)
{
if(name.empty())
{
return false;
}
creators_t::iterator it = sCreatorFunctors.find(name);
if(it != sCreatorFunctors.end())
{
//(*it).second->discard();
sCreatorFunctors.erase(it);
return true;
}
return false;
}

185
indra/llmessage/llservice.h
View File

@@ -1,185 +0,0 @@
/**
* @file llservice.h
* @author Phoenix
* @date 2004-11-21
* @brief Declaration file for LLService and related classes.
*
* $LicenseInfo:firstyear=2004&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#ifndef LL_LLSERVICE_H
#define LL_LLSERVICE_H
#include <string>
#include <map>
//#include <boost/intrusive_ptr.hpp>
//#include <boost/shared_ptr.hpp>
//#include "llframetimer.h"
#include "lliopipe.h"
#include "llchainio.h"
#if 0
class LLServiceCreator;
/**
* intrusive pointer support
*/
namespace boost
{
void intrusive_ptr_add_ref(LLServiceCreator* p);
void intrusive_ptr_release(LLServiceCreator* p);
};
#endif
/**
* @class LLServiceCreator
* @brief This class is an abstract base class for classes which create
* new <code>LLService</code> instances.
*
* Derive classes from this class which appropriately implement the
* <code>operator()</code> and destructor.
* @see LLService
*/
#if 0
class LLServiceCreator
{
public:
typedef boost::intrusive_ptr<LLService> service_t;
virtual ~LLServiceCreator() {}
virtual service_t activate() = 0;
virtual void discard() = 0;
protected:
LLServiceCreator() : mReferenceCount(0)
{
}
private:
friend void boost::intrusive_ptr_add_ref(LLServiceCreator* p);
friend void boost::intrusive_ptr_release(LLServiceCreator* p);
U32 mReferenceCount;
};
#endif
#if 0
namespace boost
{
inline void intrusive_ptr_add_ref(LLServiceCreator* p)
{
++p->mReferenceCount;
}
inline void intrusive_ptr_release(LLServiceCreator* p)
{
if(p && 0 == --p->mReferenceCount)
{
delete p;
}
}
};
#endif
/**
* @class LLService
* @brief This class is the base class for the service classes.
* @see LLIOPipe
*
* The services map a string to a chain factory with a known interface
* at the front of the chain. So, to activate a service, call
* <code>activate()</code> with the name of the service needed which
* will call the associated factory, and return a pointer to the
* known interface.
* <b>NOTE:</b> If you are implementing a service factory, it is
* vitally important that the service pipe is at the front of the
* chain.
*/
class LLService : public LLIOPipe
{
public:
//typedef boost::intrusive_ptr<LLServiceCreator> creator_t;
//typedef boost::intrusive_ptr<LLService> service_t;
typedef boost::shared_ptr<LLChainIOFactory> creator_t;
/**
* @brief This method is used to register a protocol name with a
* a functor that creates the service.
*
* THOROUGH_DESCRIPTION
* @param aParameter A brief description of aParameter.
* @return Returns true if a service was successfully registered.
*/
static bool registerCreator(const std::string& name, creator_t fn);
/**
* @brief This method connects to a service by name.
*
* @param name The name of the service to connect to.
* @param chain The constructed chain including the service instance.
* @param context Context for the activation.
* @return An instance of the service for use or NULL on failure.
*/
static LLIOPipe* activate(
const std::string& name,
LLPumpIO::chain_t& chain,
LLSD context);
/**
* @brief
*
* @param name The name of the service to discard.
* @return true if service creator was found and discarded.
*/
static bool discard(const std::string& name);
protected:
// The creation factory static data.
typedef std::map<std::string, creator_t> creators_t;
static creators_t sCreatorFunctors;
protected:
// construction & destruction. since this class is an abstract
// base class, it is up to Service implementations to actually
// deal with construction and not a public method. How that
// construction takes place will be handled by the service
// creators.
LLService();
virtual ~LLService();
protected:
// This frame timer records how long this service has
// existed. Useful for derived services to give themselves a
// lifetime and expiration.
// *NOTE: Phoenix - This functionaity has been moved to the
// pump. 2005-12-13
//LLFrameTimer mTimer;
// Since services are designed in an 'ask now, respond later'
// idiom which probably crosses thread boundaries, almost all
// services will need a handle to a response pipe. It will usually
// be the job of the service author to derive a useful
// implementation of response, and up to the service subscriber to
// further derive that class to do something useful when the
// response comes in.
LLIOPipe::ptr_t mResponse;
};
#endif // LL_LLSERVICE_H

236
indra/llmessage/llservicebuilder.cpp
View File

@@ -1,236 +0,0 @@
/**
* @file llservicebuilder.cpp
* @brief Implementation of the LLServiceBuilder class.
*
* $LicenseInfo:firstyear=2007&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#include "linden_common.h"
#include "llapp.h"
#include "llfile.h"
#include "llservicebuilder.h"
#include "llsd.h"
#include "llsdserialize.h"
void LLServiceBuilder::loadServiceDefinitionsFromFile(
const std::string& service_filename)
{
llifstream service_file(service_filename, std::ios::binary);
if(service_file.is_open())
{
LLSD service_data;
LLSDSerialize::fromXMLDocument(service_data, service_file);
service_file.close();
// Load service
LLSD service_map = service_data["services"];
for(LLSD::array_iterator array_itr = service_map.beginArray();
array_itr != service_map.endArray();
++array_itr)
{
LLSD service_llsd = (*array_itr)["service-builder"];
std::string service_name = (*array_itr)["name"].asString();
createServiceDefinition(service_name, service_llsd);
}
llinfos << "loaded config file: " << service_filename << llendl;
}
else
{
llwarns << "unable to find config file: " << service_filename << llendl;
}
}
void LLServiceBuilder::createServiceDefinition(
const std::string& service_name,
LLSD& service_llsd)
{
if(service_llsd.isString())
{
mServiceMap[ service_name ] = service_llsd.asString();
}
else if(service_llsd.isMap())
{
for(LLSD::map_iterator map_itr = service_llsd.beginMap();
map_itr != service_llsd.endMap();
++map_itr)
{
std::string compound_builder_name = service_name;
compound_builder_name.append("-");
compound_builder_name.append((*map_itr).first);
mServiceMap[ compound_builder_name ] = (*map_itr).second.asString();
}
}
}
static
bool starts_with(const std::string& text, const char* prefix)
{
return text.substr(0, strlen(prefix)) == prefix;
}
// TODO: Build a real services.xml for windows development.
// and remove the base_url logic below.
std::string LLServiceBuilder::buildServiceURI(const std::string& service_name) const
{
std::ostringstream service_url;
// Find the service builder
std::map<std::string, std::string>::const_iterator it =
mServiceMap.find(service_name);
if(it != mServiceMap.end())
{
// construct the service builder url
LLApp* app = LLApp::instance();
if(app)
{
// We define a base-url for some development configurations
// In production neither of these are defined and all services have full urls
LLSD base_url;
if (starts_with(service_name,"cap"))
{
base_url = app->getOption("cap-base-url");
}
if (base_url.asString().empty())
{
base_url = app->getOption("services-base-url");
}
service_url << base_url.asString();
}
service_url << it->second;
}
else
{
llwarns << "Cannot find service " << service_name << llendl;
}
return service_url.str();
}
std::string LLServiceBuilder::buildServiceURI(
const std::string& service_name,
const LLSD& option_map) const
{
return russ_format(buildServiceURI(service_name), option_map);
}
std::string russ_format(const std::string& format_str, const LLSD& context)
{
std::string service_url(format_str);
if(!service_url.empty() && context.isMap())
{
// throw in a ridiculously large limiter to make sure we don't
// loop forever with bad input.
int iterations = 100;
bool keep_looping = true;
while(keep_looping)
{
if(0 == --iterations)
{
keep_looping = false;
}
int depth = 0;
int deepest = 0;
bool find_match = false;
std::string::iterator iter(service_url.begin());
std::string::iterator end(service_url.end());
std::string::iterator deepest_node(service_url.end());
std::string::iterator deepest_node_end(service_url.end());
// parse out the variables to replace by going through {}s
// one at a time, starting with the "deepest" in series
// {{}}, and otherwise replacing right-to-left
for(; iter != end; ++iter)
{
switch(*iter)
{
case '{':
++depth;
if(depth > deepest)
{
deepest = depth;
deepest_node = iter;
find_match = true;
}
break;
case '}':
--depth;
if(find_match)
{
deepest_node_end = iter;
find_match = false;
}
break;
default:
break;
}
}
if((deepest_node == end) || (deepest_node_end == end))
{
break;
}
//replace the variable we found in the {} above.
// *NOTE: since the c++ implementation only understands
// params and straight string substitution, so it's a
// known distance of 2 to skip the directive.
std::string key(deepest_node + 2, deepest_node_end);
LLSD value = context[key];
switch(*(deepest_node + 1))
{
case '$':
if(value.isDefined())
{
service_url.replace(
deepest_node,
deepest_node_end + 1,
value.asString());
}
else
{
llwarns << "Unknown key: " << key << " in option map: "
<< LLSDOStreamer<LLSDNotationFormatter>(context)
<< llendl;
keep_looping = false;
}
break;
case '%':
{
std::string query_str = LLURI::mapToQueryString(value);
service_url.replace(
deepest_node,
deepest_node_end + 1,
query_str);
}
break;
default:
llinfos << "Unknown directive: " << *(deepest_node + 1)
<< llendl;
keep_looping = false;
break;
}
}
}
if (service_url.find('{') != std::string::npos)
{
llwarns << "Constructed a likely bogus service URL: " << service_url
<< llendl;
}
return service_url;
}

104
indra/llmessage/llservicebuilder.h
View File

@@ -1,104 +0,0 @@
/**
* @file llservicebuilder.h
* @brief Declaration of the LLServiceBuilder class.
*
* $LicenseInfo:firstyear=2007&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation;
* version 2.1 of the License only.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*
* Linden Research, Inc., 945 Battery Street, San Francisco, CA 94111 USA
* $/LicenseInfo$
*/
#ifndef LLSERVICEBUILDER_H
#define LLSERVICEBUILDER_H
#include <string>
#include <map>
#include "llerror.h"
class LLSD;
/**
* @brief Format format string according to rules for RUSS.
*
* This function appears alongside the service builder since the
* algorithm was originally implemented there. This can eventually be
* moved when someone wants to take the time.
* @see https://osiris.lindenlab.com/mediawiki/index.php/Recursive_URL_Substitution_Syntax
* @param format_str The input string to format.
* @param context A map used for string substitutions.
* @return Returns the formatted string. If no match is found for a
* substitution target, the braces remain intact.
*/
std::string russ_format(const std::string& format_str, const LLSD& context);
/**
* @class LLServiceBuilder
* @brief This class builds urls for us to use when making web service calls.
*/
class LLServiceBuilder
{
LOG_CLASS(LLServiceBuilder);
public:
LLServiceBuilder(void) {}
~LLServiceBuilder(void) {}
/**
* @brief Initialize this object with the service definitions.
*
* @param service_filename The services definition files -- services.xml.
*/
void loadServiceDefinitionsFromFile(const std::string& service_filename);
/**
* @brief Build a service url if the url needs no construction parameters.
*
* @param service_name The name of the service you want to call.
*/
std::string buildServiceURI(const std::string& service_name) const;
/**
* @brief Build a service url if the url with construction parameters.
*
* The parameter substitution supports string substituition from RUSS:
* [[Recursive_URL_Substitution_Syntax]]
* @param service_name The name of the service you want to call.
* @param option_map The parameters in a map of name:value for the service.
*/
std::string buildServiceURI(
const std::string& service_name,
const LLSD& option_map) const;
public:
/**
* @brief Helper method which builds construction state for a service
*
* This method should probably be protected, but we need to test this
* method.
*/
void createServiceDefinition(
const std::string& service_name,
LLSD& service_url);
protected:
std::map<std::string, std::string> mServiceMap;
};
#endif

35
indra/llmessage/lltemplatemessagereader.cpp
View File

@@ -443,7 +443,7 @@ S32 LLTemplateMessageReader::getMessageSize() const
// Returns template for the message contained in buffer
BOOL LLTemplateMessageReader::decodeTemplate(
const U8* buffer, S32 buffer_size, // inputs
LLMessageTemplate** msg_template ) // outputs
LLMessageTemplate** msg_template, bool custom ) // outputs
{
const U8* header = buffer + LL_PACKET_ID_SIZE;
@@ -485,8 +485,9 @@ BOOL LLTemplateMessageReader::decodeTemplate(
}
else // bogus packet received (too short)
{
llwarns << "Packet with unusable length received (too short): "
<< buffer_size << llendl;
if(!custom)
llwarns << "Packet with unusable length received (too short): "
<< buffer_size << llendl;
return(FALSE);
}
@@ -497,9 +498,11 @@ BOOL LLTemplateMessageReader::decodeTemplate(
}
else
{
llwarns << "Message #" << std::hex << num << std::dec
<< " received but not registered!" << llendl;
gMessageSystem->callExceptionFunc(MX_UNREGISTERED_MESSAGE);
if(!custom) {
llwarns << "Message #" << std::hex << num << std::dec
<< " received but not registered!" << llendl;
gMessageSystem->callExceptionFunc(MX_UNREGISTERED_MESSAGE);
}
return(FALSE);
}
@@ -528,7 +531,7 @@ void LLTemplateMessageReader::logRanOffEndOfPacket( const LLHost& host, const S3
static LLFastTimer::DeclareTimer FTM_PROCESS_MESSAGES("Process Messages");
// decode a given message
BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender )
BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender, bool custom)
{
llassert( mReceiveSize >= 0 );
llassert( mCurrentRMessageTemplate);
@@ -586,7 +589,8 @@ BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender
}
else
{
llerrs << "Unknown block type" << llendl;
if(!custom)
llerrs << "Unknown block type" << llendl;
return FALSE;
}
@@ -632,7 +636,8 @@ BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender
if ((decode_pos + data_size) > mReceiveSize)
{
logRanOffEndOfPacket(sender, decode_pos, data_size);
if (!custom)
logRanOffEndOfPacket(sender, decode_pos, data_size);
// default to 0 length variable blocks
tsize = 0;
@@ -668,7 +673,8 @@ BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender
// so, copy data pointer and set data size to fixed size
if ((decode_pos + mvci.getSize()) > mReceiveSize)
{
logRanOffEndOfPacket(sender, decode_pos, mvci.getSize());
if(!custom)
logRanOffEndOfPacket(sender, decode_pos, mvci.getSize());
// default to 0s.
U32 size = mvci.getSize();
@@ -696,6 +702,7 @@ BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender
return FALSE;
}
if(!custom)
{
static LLTimer decode_timer;
@@ -751,11 +758,11 @@ BOOL LLTemplateMessageReader::decodeData(const U8* buffer, const LLHost& sender
BOOL LLTemplateMessageReader::validateMessage(const U8* buffer,
S32 buffer_size,
const LLHost& sender,
bool trusted)
bool trusted, bool custom)
{
mReceiveSize = buffer_size;
BOOL valid = decodeTemplate(buffer, buffer_size, &mCurrentRMessageTemplate );
if(valid)
BOOL valid = decodeTemplate(buffer, buffer_size, &mCurrentRMessageTemplate, custom);
if(valid && !custom)
{
mCurrentRMessageTemplate->mReceiveCount++;
//lldebugs << "MessageRecvd:"
@@ -787,7 +794,7 @@ BOOL LLTemplateMessageReader::validateMessage(const U8* buffer,
BOOL LLTemplateMessageReader::readMessage(const U8* buffer,
const LLHost& sender)
{
return decodeData(buffer, sender);
return decodeData(buffer, sender, false);
}
//virtual

8
indra/llmessage/lltemplatemessagereader.h
View File

@@ -99,7 +99,7 @@ public:
virtual void copyToBuilder(LLMessageBuilder&) const;
BOOL validateMessage(const U8* buffer, S32 buffer_size,
const LLHost& sender, bool trusted = false);
const LLHost& sender, bool trusted = false, bool custom = false);
BOOL readMessage(const U8* buffer, const LLHost& sender);
bool isTrusted() const;
@@ -112,16 +112,18 @@ private:
S32 size = 0, S32 blocknum = 0, S32 max_size = S32_MAX);
BOOL decodeTemplate(const U8* buffer, S32 buffer_size, // inputs
LLMessageTemplate** msg_template ); // outputs
LLMessageTemplate** msg_template, // outputs
bool custom = false);
void logRanOffEndOfPacket( const LLHost& host, const S32 where, const S32 wanted );
BOOL decodeData(const U8* buffer, const LLHost& sender );
BOOL decodeData(const U8* buffer, const LLHost& sender, bool custom);
S32 mReceiveSize;
LLMessageTemplate* mCurrentRMessageTemplate;
LLMsgData* mCurrentRMessageData;
message_template_number_map_t& mMessageNumbers;
friend class LLFloaterMessageLogItem;
};
#endif // LL_LLTEMPLATEMESSAGEREADER_H

660
indra/llmessage/llurlrequest.cpp
View File

@@ -36,8 +36,7 @@
#include <algorithm>
#include <openssl/x509_vfy.h>
#include <openssl/ssl.h>
#include "llcurl.h"
#include "aicurleasyrequeststatemachine.h"
#include "llioutil.h"
#include "llmemtype.h"
#include "llpumpio.h"
@@ -47,6 +46,7 @@
#include "llapr.h"
#include "llscopedvolatileaprpool.h"
#include "llfasttimer.h"
#include "message.h"
static const U32 HTTP_STATUS_PIPE_ERROR = 499;
/**
@@ -54,41 +54,6 @@ static const U32 HTTP_STATUS_PIPE_ERROR = 499;
*/
const std::string CONTEXT_TRANSFERED_BYTES("transfered_bytes");
static size_t headerCallback(char* data, size_t size, size_t nmemb, void* user);
/**
* class LLURLRequestDetail
*/
class LLURLRequestDetail
{
public:
LLURLRequestDetail();
~LLURLRequestDetail();
std::string mURL;
AICurlEasyRequest mCurlEasyRequest;
LLIOPipe::buffer_ptr_t mResponseBuffer;
LLChannelDescriptors mChannels;
U8* mLastRead;
U32 mBodyLimit;
S32 mByteAccumulator;
bool mIsBodyLimitSet;
};
LLURLRequestDetail::LLURLRequestDetail() :
mCurlEasyRequest(false),
mLastRead(NULL),
mBodyLimit(0),
mByteAccumulator(0),
mIsBodyLimitSet(false)
{
}
LLURLRequestDetail::~LLURLRequestDetail()
{
mLastRead = NULL;
}
/**
* class LLURLRequest
*/
@@ -96,7 +61,8 @@ LLURLRequestDetail::~LLURLRequestDetail()
// static
std::string LLURLRequest::actionAsVerb(LLURLRequest::ERequestAction action)
{
static const std::string VERBS[] =
static int const array_size = HTTP_MOVE + 1; // INVALID == 0
static char const* const VERBS[array_size] =
{
"(invalid)",
"HEAD",
@@ -106,81 +72,88 @@ std::string LLURLRequest::actionAsVerb(LLURLRequest::ERequestAction action)
"DELETE",
"MOVE"
};
if(((S32)action <=0) || ((S32)action >= REQUEST_ACTION_COUNT))
return VERBS[action >= array_size ? INVALID : action];
}
// This might throw AICurlNoEasyHandle.
LLURLRequest::LLURLRequest(LLURLRequest::ERequestAction action, std::string const& url, Injector* body,
LLHTTPClient::ResponderPtr responder, AIHTTPHeaders& headers, bool is_auth, bool no_compression) :
mAction(action), mURL(url), mIsAuth(is_auth), mNoCompression(no_compression),
mBody(body), mResponder(responder), mHeaders(headers)
{
}
void LLURLRequest::initialize_impl(void)
{
if (mHeaders.hasHeader("Cookie"))
{
return VERBS[0];
allowCookies();
}
return VERBS[action];
}
LLURLRequest::LLURLRequest(LLURLRequest::ERequestAction action) :
mAction(action)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
// This might throw AICurlNoEasyHandle.
initialize();
}
LLURLRequest::LLURLRequest(
LLURLRequest::ERequestAction action,
const std::string& url) :
mAction(action)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
// This might throw AICurlNoEasyHandle.
initialize();
setURL(url);
}
LLURLRequest::~LLURLRequest()
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
// If the header is "Pragma" with no value, the caller intends to
// force libcurl to drop the Pragma header it so gratuitously inserts.
// Before inserting the header, force libcurl to not use the proxy.
std::string pragma_value;
if (mHeaders.getValue("Pragma", pragma_value) && pragma_value.empty())
{
AICurlEasyRequest_wat curl_easy_request_w(*mDetail->mCurlEasyRequest);
curl_easy_request_w->revokeCallbacks();
curl_easy_request_w->send_events_to(NULL);
useProxy(false);
}
delete mDetail;
}
void LLURLRequest::setURL(const std::string& url)
{
mDetail->mURL = url;
}
if (mAction == HTTP_PUT || mAction == HTTP_POST)
{
// If the Content-Type header was passed in we defer to the caller's wisdom,
// but if they did not specify a Content-Type, then ask the injector.
mHeaders.addHeader("Content-Type", mBody->contentType(), AIHTTPHeaders::keep_existing_header);
}
else
{
// Check to see if we have already set Accept or not. If no one
// set it, set it to application/llsd+xml since that's what we
// almost always want.
mHeaders.addHeader("Accept", "application/llsd+xml", AIHTTPHeaders::keep_existing_header);
}
std::string LLURLRequest::getURL() const
{
return mDetail->mURL;
if (mAction == HTTP_POST && gMessageSystem)
{
mHeaders.addHeader("X-SecondLife-UDP-Listen-Port", llformat("%d", gMessageSystem->mPort));
}
bool success = false;
try
{
AICurlEasyRequest_wat easy_request_w(*mCurlEasyRequest);
easy_request_w->prepRequest(easy_request_w, mHeaders, mResponder);
if (mBody)
{
// This might throw AICurlNoBody.
mBodySize = mBody->get_body(easy_request_w->sChannels, easy_request_w->getInput());
}
success = configure(easy_request_w);
}
catch (AICurlNoBody const& error)
{
llwarns << "Injector::get_body() failed: " << error.what() << llendl;
}
if (success)
{
// Continue to initialize base class.
AICurlEasyRequestStateMachine::initialize_impl();
}
else
{
abort();
}
}
void LLURLRequest::addHeader(const char* header)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
AICurlEasyRequest_wat curlEasyRequest_w(*mCurlEasyRequest);
curlEasyRequest_w->addHeader(header);
}
void LLURLRequest::checkRootCertificate(bool check)
{
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
curlEasyRequest_w->setopt(CURLOPT_SSL_VERIFYPEER, check ? 1L : 0L);
curlEasyRequest_w->setoptString(CURLOPT_ENCODING, "");
}
void LLURLRequest::setBodyLimit(U32 size)
{
mDetail->mBodyLimit = size;
mDetail->mIsBodyLimitSet = true;
}
void LLURLRequest::setCallback(LLURLRequestComplete* callback)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
mCompletionCallback = callback;
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
curlEasyRequest_w->setHeaderCallback(&headerCallback, (void*)callback);
}
// Added to mitigate the effect of libcurl looking
// for the ALL_PROXY and http_proxy env variables
// and deciding to insert a Pragma: no-cache
@@ -214,288 +187,31 @@ void LLURLRequest::useProxy(bool use_proxy)
LL_DEBUGS("Proxy") << "use_proxy = " << (use_proxy?'Y':'N') << ", env_proxy = " << (!env_proxy.empty() ? env_proxy : "(null)") << LL_ENDL;
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
AICurlEasyRequest_wat curlEasyRequest_w(*mCurlEasyRequest);
curlEasyRequest_w->setoptString(CURLOPT_PROXY, (use_proxy && !env_proxy.empty()) ? env_proxy : std::string(""));
}
#ifdef AI_UNUSED
void LLURLRequest::useProxy(const std::string &proxy)
{
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
AICurlEasyRequest_wat curlEasyRequest_w(*mCurlEasyRequest);
curlEasyRequest_w->setoptString(CURLOPT_PROXY, proxy);
}
#endif
void LLURLRequest::allowCookies()
{
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
AICurlEasyRequest_wat curlEasyRequest_w(*mCurlEasyRequest);
curlEasyRequest_w->setoptString(CURLOPT_COOKIEFILE, "");
}
//virtual
bool LLURLRequest::hasExpiration(void) const
bool LLURLRequest::configure(AICurlEasyRequest_wat const& curlEasyRequest_w)
{
// Currently, this ALWAYS returns false -- because only AICurlEasyRequestStateMachine uses buffered
// AICurlEasyRequest objects, and LLURLRequest uses (unbuffered) AICurlEasyRequest directly, which
// have no expiration facility.
return mDetail->mCurlEasyRequest.isBuffered();
}
//virtual
bool LLURLRequest::hasNotExpired(void) const
{
if (!mDetail->mCurlEasyRequest.isBuffered())
return true;
AICurlEasyRequest_wat buffered_easy_request_w(*mDetail->mCurlEasyRequest);
AICurlResponderBuffer_wat buffer_w(*mDetail->mCurlEasyRequest);
return buffer_w->isValid();
}
// virtual
LLIOPipe::EStatus LLURLRequest::handleError(
LLIOPipe::EStatus status,
LLPumpIO* pump)
{
DoutEntering(dc::curl, "LLURLRequest::handleError(" << LLIOPipe::lookupStatusString(status) << ", " << (void*)pump << ")");
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
if (LL_LIKELY(!mDetail->mCurlEasyRequest.isBuffered())) // Currently always true.
{
// The last reference will be deleted when the pump that this chain belongs to
// is removed from the running chains vector, upon returning from this function.
// This keeps the CurlEasyRequest object alive until the curl thread cleanly removed it.
Dout(dc::curl, "Calling mDetail->mCurlEasyRequest.removeRequest()");
mDetail->mCurlEasyRequest.removeRequest();
}
else if (!hasNotExpired())
{
// The buffered version has it's own time out handling, and that already expired,
// so we can ignore the expiration of this timer (currently never happens).
// I left it here because it's what LL did (in the form if (!isValid() ...),
// and it would be relevant if this characteristic of mDetail->mCurlEasyRequest
// would change. --Aleric
return STATUS_EXPIRED ;
}
if(mCompletionCallback && pump)
{
LLURLRequestComplete* complete = NULL;
complete = (LLURLRequestComplete*)mCompletionCallback.get();
complete->httpStatus(
HTTP_STATUS_PIPE_ERROR,
LLIOPipe::lookupStatusString(status));
complete->responseStatus(status);
pump->respond(complete);
mCompletionCallback = NULL;
}
return status;
}
void LLURLRequest::added_to_multi_handle(AICurlEasyRequest_wat&)
{
}
void LLURLRequest::finished(AICurlEasyRequest_wat&)
{
}
void LLURLRequest::removed_from_multi_handle(AICurlEasyRequest_wat&)
{
mRemoved = true;
}
static LLFastTimer::DeclareTimer FTM_PROCESS_URL_REQUEST("URL Request");
// virtual
LLIOPipe::EStatus LLURLRequest::process_impl(
const LLChannelDescriptors& channels,
buffer_ptr_t& buffer,
bool& eos,
LLSD& context,
LLPumpIO* pump)
{
LLFastTimer t(FTM_PROCESS_URL_REQUEST);
PUMP_DEBUG;
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
//llinfos << "LLURLRequest::process_impl()" << llendl;
if (!buffer) return STATUS_ERROR;
if (!mDetail) return STATUS_ERROR; //Seems to happen on occasion. Need to hunt down why.
// we're still waiting or processing, check how many
// bytes we have accumulated.
const S32 MIN_ACCUMULATION = 100000;
if(pump && (mDetail->mByteAccumulator > MIN_ACCUMULATION))
{
static LLFastTimer::DeclareTimer FTM_URL_ADJUST_TIMEOUT("Adjust Timeout");
LLFastTimer t(FTM_URL_ADJUST_TIMEOUT);
// This is a pretty sloppy calculation, but this
// tries to make the gross assumption that if data
// is coming in at 56kb/s, then this transfer will
// probably succeed. So, if we're accumlated
// 100,000 bytes (MIN_ACCUMULATION) then let's
// give this client another 2s to complete.
const F32 TIMEOUT_ADJUSTMENT = 2.0f;
mDetail->mByteAccumulator = 0;
pump->adjustTimeoutSeconds(TIMEOUT_ADJUSTMENT);
lldebugs << "LLURLRequest adjustTimeoutSeconds for request: " << mDetail->mURL << llendl;
if (mState == STATE_INITIALIZED)
{
llinfos << "LLURLRequest adjustTimeoutSeconds called during upload" << llendl;
}
}
switch(mState)
{
case STATE_INITIALIZED:
{
PUMP_DEBUG;
// We only need to wait for input if we are uploading
// something.
if(((HTTP_PUT == mAction) || (HTTP_POST == mAction)) && !eos)
{
// we're waiting to get all of the information
return STATUS_BREAK;
}
// *FIX: bit of a hack, but it should work. The configure and
// callback method expect this information to be ready.
mDetail->mResponseBuffer = buffer;
mDetail->mChannels = channels;
if(!configure())
{
return STATUS_ERROR;
}
mRemoved = false;
mState = STATE_WAITING_FOR_RESPONSE;
mDetail->mCurlEasyRequest.addRequest(); // Add easy handle to multi handle.
return STATUS_BREAK;
}
case STATE_WAITING_FOR_RESPONSE:
case STATE_PROCESSING_RESPONSE:
{
if (!mRemoved) // Not removed from multi handle yet?
{
// Easy handle is still being processed.
return STATUS_BREAK;
}
// Curl thread finished with this easy handle.
mState = STATE_CURL_FINISHED;
}
case STATE_CURL_FINISHED:
{
PUMP_DEBUG;
LLIOPipe::EStatus status = STATUS_NO_CONNECTION; // Catch-all failure code.
// Left braces in order not to change indentation.
{
CURLcode result;
static LLFastTimer::DeclareTimer FTM_PROCESS_URL_REQUEST_GET_RESULT("Get Result");
AICurlEasyRequest_wat(*mDetail->mCurlEasyRequest)->getResult(&result);
mState = STATE_HAVE_RESPONSE;
context[CONTEXT_REQUEST][CONTEXT_TRANSFERED_BYTES] = mRequestTransferedBytes;
context[CONTEXT_RESPONSE][CONTEXT_TRANSFERED_BYTES] = mResponseTransferedBytes;
lldebugs << this << "Setting context to " << context << llendl;
switch(result)
{
case CURLE_OK:
case CURLE_WRITE_ERROR:
// NB: The error indication means that we stopped the
// writing due the body limit being reached
if(mCompletionCallback && pump)
{
LLURLRequestComplete* complete = NULL;
complete = (LLURLRequestComplete*)
mCompletionCallback.get();
complete->responseStatus(
result == CURLE_OK
? STATUS_OK : STATUS_STOP);
LLPumpIO::links_t chain;
LLPumpIO::LLLinkInfo link;
link.mPipe = mCompletionCallback;
link.mChannels = LLBufferArray::makeChannelConsumer(
channels);
chain.push_back(link);
static LLFastTimer::DeclareTimer FTM_PROCESS_URL_PUMP_RESPOND("Pump Respond");
{
LLFastTimer t(FTM_PROCESS_URL_PUMP_RESPOND);
pump->respond(chain, buffer, context);
}
mCompletionCallback = NULL;
}
status = STATUS_BREAK; // This is what the old code returned. Does it make sense?
break;
case CURLE_FAILED_INIT:
case CURLE_COULDNT_CONNECT:
status = STATUS_NO_CONNECTION;
break;
default:
llwarns << "URLRequest Error: " << result
<< ", "
<< LLCurl::strerror(result)
<< ", "
<< (mDetail->mURL.empty() ? "<EMPTY URL>" : mDetail->mURL)
<< llendl;
status = STATUS_ERROR;
break;
}
}
return status;
}
case STATE_HAVE_RESPONSE:
PUMP_DEBUG;
// we already stuffed everything into channel in in the curl
// callback, so we are done.
eos = true;
context[CONTEXT_REQUEST][CONTEXT_TRANSFERED_BYTES] = mRequestTransferedBytes;
context[CONTEXT_RESPONSE][CONTEXT_TRANSFERED_BYTES] = mResponseTransferedBytes;
lldebugs << this << "Setting context to " << context << llendl;
return STATUS_DONE;
default:
PUMP_DEBUG;
context[CONTEXT_REQUEST][CONTEXT_TRANSFERED_BYTES] = mRequestTransferedBytes;
context[CONTEXT_RESPONSE][CONTEXT_TRANSFERED_BYTES] = mResponseTransferedBytes;
lldebugs << this << "Setting context to " << context << llendl;
return STATUS_ERROR;
}
}
void LLURLRequest::initialize()
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
mState = STATE_INITIALIZED;
// This might throw AICurlNoEasyHandle.
mDetail = new LLURLRequestDetail;
{
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
curlEasyRequest_w->setWriteCallback(&downCallback, (void*)this);
curlEasyRequest_w->setReadCallback(&upCallback, (void*)this);
}
mRequestTransferedBytes = 0;
mResponseTransferedBytes = 0;
}
static LLFastTimer::DeclareTimer FTM_URL_REQUEST_CONFIGURE("URL Configure");
bool LLURLRequest::configure()
{
LLFastTimer t(FTM_URL_REQUEST_CONFIGURE);
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
bool rv = false;
S32 bytes = mDetail->mResponseBuffer->countAfter(
mDetail->mChannels.in(),
NULL);
{
AICurlEasyRequest_wat curlEasyRequest_w(*mDetail->mCurlEasyRequest);
switch(mAction)
{
case HTTP_HEAD:
curlEasyRequest_w->setopt(CURLOPT_HEADER, 1);
curlEasyRequest_w->setopt(CURLOPT_NOBODY, 1);
curlEasyRequest_w->setopt(CURLOPT_FOLLOWLOCATION, 1);
rv = true;
@@ -506,28 +222,30 @@ bool LLURLRequest::configure()
curlEasyRequest_w->setopt(CURLOPT_FOLLOWLOCATION, 1);
// Set Accept-Encoding to allow response compression
curlEasyRequest_w->setoptString(CURLOPT_ENCODING, "");
curlEasyRequest_w->setoptString(CURLOPT_ENCODING, mNoCompression ? "identity" : "");
rv = true;
break;
case HTTP_PUT:
{
// Disable the expect http 1.1 extension. POST and PUT default
// to turning this on, and I am not too sure what it means.
// to using this, causing the broken server to get confused.
curlEasyRequest_w->addHeader("Expect:");
curlEasyRequest_w->setopt(CURLOPT_UPLOAD, 1);
curlEasyRequest_w->setopt(CURLOPT_INFILESIZE, bytes);
curlEasyRequest_w->setopt(CURLOPT_INFILESIZE, mBodySize);
rv = true;
break;
}
case HTTP_POST:
{
// Set the handle for an http post
curlEasyRequest_w->setPost(bytes);
curlEasyRequest_w->setPost(mBodySize);
// Set Accept-Encoding to allow response compression
curlEasyRequest_w->setoptString(CURLOPT_ENCODING, "");
curlEasyRequest_w->setoptString(CURLOPT_ENCODING, mNoCompression ? "identity" : "");
rv = true;
break;
}
case HTTP_DELETE:
// Set the handle for an http post
curlEasyRequest_w->setoptString(CURLOPT_CUSTOMREQUEST, "DELETE");
@@ -537,7 +255,6 @@ bool LLURLRequest::configure()
case HTTP_MOVE:
// Set the handle for an http post
curlEasyRequest_w->setoptString(CURLOPT_CUSTOMREQUEST, "MOVE");
// *NOTE: should we check for the Destination header?
rv = true;
break;
@@ -547,205 +264,12 @@ bool LLURLRequest::configure()
}
if(rv)
{
curlEasyRequest_w->finalizeRequest(mDetail->mURL);
curlEasyRequest_w->send_events_to(this);
curlEasyRequest_w->setopt(CURLOPT_SSL_VERIFYPEER, gNoVerifySSLCert ? 0L : 1L);
// Don't verify host name if this is not an authentication request,
// so urls with scrubbed host names will work (improves DNS performance).
curlEasyRequest_w->setopt(CURLOPT_SSL_VERIFYHOST, (gNoVerifySSLCert || !mIsAuth) ? 0L : 2L);
curlEasyRequest_w->finalizeRequest(mURL, mResponder->getHTTPTimeoutPolicy(), this);
}
}
return rv;
}
// static
size_t LLURLRequest::downCallback(
char* data,
size_t size,
size_t nmemb,
void* user)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
LLURLRequest* req = (LLURLRequest*)user;
if(STATE_WAITING_FOR_RESPONSE == req->mState)
{
req->mState = STATE_PROCESSING_RESPONSE;
}
U32 bytes = size * nmemb;
if (req->mDetail->mIsBodyLimitSet)
{
if (bytes > req->mDetail->mBodyLimit)
{
bytes = req->mDetail->mBodyLimit;
req->mDetail->mBodyLimit = 0;
}
else
{
req->mDetail->mBodyLimit -= bytes;
}
}
req->mDetail->mResponseBuffer->append(
req->mDetail->mChannels.out(),
(U8*)data,
bytes);
req->mResponseTransferedBytes += bytes;
req->mDetail->mByteAccumulator += bytes;
return bytes;
}
// static
size_t LLURLRequest::upCallback(
char* data,
size_t size,
size_t nmemb,
void* user)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
LLURLRequest* req = (LLURLRequest*)user;
S32 bytes = llmin(
(S32)(size * nmemb),
req->mDetail->mResponseBuffer->countAfter(
req->mDetail->mChannels.in(),
req->mDetail->mLastRead));
req->mDetail->mLastRead = req->mDetail->mResponseBuffer->readAfter(
req->mDetail->mChannels.in(),
req->mDetail->mLastRead,
(U8*)data,
bytes);
req->mRequestTransferedBytes += bytes;
return bytes;
}
static size_t headerCallback(char* header_line, size_t size, size_t nmemb, void* user)
{
size_t header_len = size * nmemb;
LLURLRequestComplete* complete = (LLURLRequestComplete*)user;
if (!complete || !header_line)
{
return header_len;
}
// *TODO: This should be a utility in llstring.h: isascii()
for (size_t i = 0; i < header_len; ++i)
{
if (header_line[i] < 0)
{
return header_len;
}
}
std::string header(header_line, header_len);
// Per HTTP spec the first header line must be the status line.
if (header.substr(0,5) == "HTTP/")
{
std::string::iterator end = header.end();
std::string::iterator pos1 = std::find(header.begin(), end, ' ');
if (pos1 != end) ++pos1;
std::string::iterator pos2 = std::find(pos1, end, ' ');
if (pos2 != end) ++pos2;
std::string::iterator pos3 = std::find(pos2, end, '\r');
std::string version(header.begin(), pos1);
std::string status(pos1, pos2);
std::string reason(pos2, pos3);
S32 status_code = atoi(status.c_str());
if (status_code > 0)
{
complete->httpStatus((U32)status_code, reason);
return header_len;
}
}
std::string::iterator sep = std::find(header.begin(),header.end(),':');
if (sep != header.end())
{
std::string key(header.begin(), sep);
std::string value(sep + 1, header.end());
key = utf8str_tolower(utf8str_trim(key));
value = utf8str_trim(value);
complete->header(key, value);
}
else
{
LLStringUtil::trim(header);
if (!header.empty())
{
llwarns << "Unable to parse header: " << header << llendl;
}
}
return header_len;
}
/**
* LLURLRequestComplete
*/
LLURLRequestComplete::LLURLRequestComplete() :
mRequestStatus(LLIOPipe::STATUS_ERROR)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
}
// virtual
LLURLRequestComplete::~LLURLRequestComplete()
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
}
//virtual
void LLURLRequestComplete::header(const std::string& header, const std::string& value)
{
}
//virtual
void LLURLRequestComplete::complete(const LLChannelDescriptors& channels,
const buffer_ptr_t& buffer)
{
if(STATUS_OK == mRequestStatus)
{
response(channels, buffer);
}
else
{
noResponse();
}
}
//virtual
void LLURLRequestComplete::response(const LLChannelDescriptors& channels,
const buffer_ptr_t& buffer)
{
llwarns << "LLURLRequestComplete::response default implementation called"
<< llendl;
}
//virtual
void LLURLRequestComplete::noResponse()
{
llwarns << "LLURLRequestComplete::noResponse default implementation called"
<< llendl;
}
void LLURLRequestComplete::responseStatus(LLIOPipe::EStatus status)
{
LLMemType m1(LLMemType::MTYPE_IO_URL_REQUEST);
mRequestStatus = status;
}
static LLFastTimer::DeclareTimer FTM_PROCESS_URL_COMPLETE("URL Complete");
// virtual
LLIOPipe::EStatus LLURLRequestComplete::process_impl(
const LLChannelDescriptors& channels,
buffer_ptr_t& buffer,
bool& eos,
LLSD& context,
LLPumpIO* pump)
{
LLFastTimer t(FTM_PROCESS_URL_COMPLETE);
PUMP_DEBUG;
complete(channels, buffer);
return STATUS_OK;
}

300
indra/llmessage/llurlrequest.h
View File

@@ -7,6 +7,7 @@
* $LicenseInfo:firstyear=2005&license=viewerlgpl$
* Second Life Viewer Source Code
* Copyright (C) 2010, Linden Research, Inc.
* Copyright (C) 2012, Aleric Inglewood.
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
@@ -35,41 +36,21 @@
*/
#include <string>
#include "lliopipe.h"
#include "llchainio.h"
#include "llerror.h"
#include "llcurl.h"
#include "aicurleasyrequeststatemachine.h"
#include "aihttpheaders.h"
extern const std::string CONTEXT_REQUEST;
extern const std::string CONTEXT_RESPONSE;
extern const std::string CONTEXT_TRANSFERED_BYTES;
class LLURLRequestDetail;
class LLURLRequestComplete;
struct x509_store_ctx_st;
typedef struct x509_store_ctx_st X509_STORE_CTX;
/**
* @class LLURLRequest
* @brief Class to handle url based requests.
* @see LLIOPipe
*
* Currently, this class is implemented on top of curl. From the
* vantage of a programmer using this class, you do not care so much,
* but it's useful to know since in order to accomplish 'non-blocking'
* behavior, we have to use a more expensive curl interface which can
* still block if the server enters a half-accepted state. It would be
* worth the time and effort to eventually port this to a raw client
* socket.
*/
class LLURLRequest : public LLIOPipe, protected AICurlEasyHandleEvents
class Injector
{
LOG_CLASS(LLURLRequest);
public:
public:
typedef LLHTTPClient::ResponderBase::buffer_ptr_t buffer_ptr_t;
virtual char const* contentType(void) const = 0;
virtual U32 get_body(LLChannelDescriptors const& channels, buffer_ptr_t& buffer) = 0;
// To avoid compiler warning.
virtual ~Injector() { }
};
typedef int (* SSLCertVerifyCallback)(X509_STORE_CTX *ctx, void *param);
class LLURLRequest : public AICurlEasyRequestStateMachine {
public:
/**
* @brief This enumeration is for specifying the type of request.
*/
@@ -86,44 +67,33 @@ public:
};
/**
* @brief Turn the requst action into an http verb.
* @brief Turn the request action into an http verb.
*/
static std::string actionAsVerb(ERequestAction action);
/**
* @brief Constructor.
*
* @param action One of the ERequestAction enumerations.
*/
LLURLRequest(ERequestAction action);
/**
* @brief Constructor.
*
* @param action One of the ERequestAction enumerations.
* @param url The url of the request. It should already be encoded.
*/
LLURLRequest(ERequestAction action, const std::string& url);
LLURLRequest(ERequestAction action, std::string const& url, Injector* body, LLHTTPClient::ResponderPtr responder, AIHTTPHeaders& headers, bool is_auth, bool no_compression);
/**
* @brief Destructor.
*/
virtual ~LLURLRequest();
protected:
// Call abort(), not delete.
/*virtual*/ ~LLURLRequest() { }
/* @name Instance methods
public:
/**
* @brief Turn on cookie handling for this request with CURLOPT_COOKIEFILE.
*/
//@{
/**
* @brief Set the url for the request
*
* This method assumes the url is encoded appropriately for the
* request.
* The url must be set somehow before the first call to process(),
* or the url will not be set correctly.
*
*/
void setURL(const std::string& url);
std::string getURL() const;
void allowCookies(void);
/**
* @ brief Turn off (or on) the CURLOPT_PROXY header.
*/
void useProxy(bool use_proxy);
/**
* @brief Add a header to the http post.
*
@@ -133,213 +103,29 @@ public:
* required headers will be automatically constructed, so this is
* usually useful for encoding parameters.
*/
void addHeader(const char* header);
/**
* @brief Check remote server certificate signed by a known root CA.
*
* Set whether request will check that remote server
* certificates are signed by a known root CA when using HTTPS.
*/
void checkRootCertificate(bool check);
/**
* @brief Return at most size bytes of body.
*
* If the body had more bytes than this limit, they will not be
* returned and the connection closed. In this case, STATUS_STOP
* will be passed to responseStatus();
*/
void setBodyLimit(U32 size);
/**
* @brief Set a completion callback for this URLRequest.
*
* The callback is added to this URLRequet's pump when either the
* entire buffer is known or an error like timeout or connection
* refused has happened. In the case of a complete transfer, this
* object builds a response chain such that the callback and the
* next process consumer get to read the output.
*
* This setup is a little fragile since the url request consumer
* might not just read the data - it may do a channel change,
* which invalidates the input to the callback, but it works well
* in practice.
*/
void setCallback(LLURLRequestComplete* callback);
//@}
/* @name LLIOPipe virtual implementations
*/
/**
* @ brief Turn off (or on) the CURLOPT_PROXY header.
*/
void useProxy(bool use_proxy);
/**
* @ brief Set the CURLOPT_PROXY header to the given value.
*/
void useProxy(const std::string& proxy);
/**
* @brief Turn on cookie handling for this request with CURLOPT_COOKIEFILE.
*/
void allowCookies();
/*virtual*/ bool hasExpiration(void) const;
/*virtual*/ bool hasNotExpired(void) const;
public:
/**
* @brief Give this pipe a chance to handle a generated error
*/
virtual EStatus handleError(EStatus status, LLPumpIO* pump);
protected:
/**
* @brief Process the data in buffer
*/
virtual EStatus process_impl(
const LLChannelDescriptors& channels,
buffer_ptr_t& buffer,
bool& eos,
LLSD& context,
LLPumpIO* pump);
//@}
protected:
enum EState
{
STATE_INITIALIZED,
STATE_WAITING_FOR_RESPONSE,
STATE_PROCESSING_RESPONSE,
STATE_CURL_FINISHED,
STATE_HAVE_RESPONSE,
};
EState mState;
ERequestAction mAction;
LLURLRequestDetail* mDetail;
LLIOPipe::ptr_t mCompletionCallback;
S32 mRequestTransferedBytes;
S32 mResponseTransferedBytes;
// mRemoved is used instead of changing mState directly, because I'm not convinced the latter is atomic.
// Set to false before adding curl request and then only tested.
// Reset in removed_from_multi_handle (by another thread), this is thread-safe.
bool mRemoved;
/*virtual*/ void added_to_multi_handle(AICurlEasyRequest_wat&);
/*virtual*/ void finished(AICurlEasyRequest_wat&);
/*virtual*/ void removed_from_multi_handle(AICurlEasyRequest_wat&);
private:
/**
* @brief Initialize the object. Called during construction.
*/
void initialize();
void addHeader(char const* header);
private:
/**
* @brief Handle action specific url request configuration.
*
* @return Returns true if this is configured.
*/
bool configure();
bool configure(AICurlEasyRequest_wat const& curlEasyRequest_w);
/**
* @brief Download callback method.
*/
static size_t downCallback(
char* data,
size_t size,
size_t nmemb,
void* user);
private:
ERequestAction mAction;
std::string mURL;
bool mIsAuth; // Set for authentication messages (login, buy land, buy currency).
bool mNoCompression; // Set to disable using gzip.
Injector* mBody; // Non-zero iff the action is HTTP_POST and HTTP_PUT.
U32 mBodySize;
LLHTTPClient::ResponderPtr mResponder;
AIHTTPHeaders mHeaders;
/**
* @brief Upload callback method.
*/
static size_t upCallback(
char* data,
size_t size,
size_t nmemb,
void* user);
/**
* @brief Declaration of unimplemented method to prevent copy
* construction.
*/
LLURLRequest(const LLURLRequest&);
};
/**
* @class LLURLRequestComplete
* @brief Class which can optionally be used with an LLURLRequest to
* get notification when the url request is complete.
*/
class LLURLRequestComplete : public LLIOPipe
{
public:
// Called once for each header received, except status lines
virtual void header(const std::string& header, const std::string& value);
// May be called more than once, particularly for redirects and proxy madness.
// Ex. a 200 for a connection to https through a proxy, followed by the "real" status
// a 3xx for a redirect followed by a "real" status, or more redirects.
virtual void httpStatus(U32 status, const std::string& reason) { }
virtual void complete(
const LLChannelDescriptors& channels,
const buffer_ptr_t& buffer);
/**
* @brief This method is called when we got a valid response.
*
* It is up to class implementers to do something useful here.
*/
virtual void response(
const LLChannelDescriptors& channels,
const buffer_ptr_t& buffer);
/**
* @brief This method is called if there was no response.
*
* It is up to class implementers to do something useful here.
*/
virtual void noResponse();
/**
* @brief This method will be called by the LLURLRequest object.
*
* If this is set to STATUS_OK or STATUS_STOP, then the transfer
* is asssumed to have worked. This will lead to calling response()
* on the next call to process(). Otherwise, this object will call
* noResponse() on the next call to process.
* @param status The status of the URLRequest.
*/
void responseStatus(EStatus status);
// constructor & destructor.
LLURLRequestComplete();
virtual ~LLURLRequestComplete();
protected:
/* @name LLIOPipe virtual implementations
*/
//@{
/**
* @brief Process the data in buffer
*/
virtual EStatus process_impl(
const LLChannelDescriptors& channels,
buffer_ptr_t& buffer,
bool& eos,
LLSD& context,
LLPumpIO* pump);
//@}
// value to note if we actually got the response. This value
// depends on correct useage from the LLURLRequest instance.
EStatus mRequestStatus;
protected:
// Handle initializing the object.
/*virtual*/ void initialize_impl(void);
};
#endif // LL_LLURLREQUEST_H

16
indra/llmessage/message.cpp
View File

@@ -83,6 +83,9 @@
#include "llmemtype.h"
#include "llpacketring.h"
class AIHTTPTimeoutPolicy;
extern AIHTTPTimeoutPolicy fnPtrResponder_timeout;
// Constants
//const char* MESSAGE_LOG_FILENAME = "message.log";
static const F32 CIRCUIT_DUMP_TIMEOUT = 30.f;
@@ -104,7 +107,7 @@ public:
namespace
{
class LLFnPtrResponder : public LLHTTPClient::Responder
class LLFnPtrResponder : public LLHTTPClient::ResponderWithResult
{
LOG_CLASS(LLFnPtrResponder);
public:
@@ -133,6 +136,8 @@ namespace
if(NULL != mCallback) mCallback(mCallbackData, LL_ERR_NOERR);
}
virtual AIHTTPTimeoutPolicy const& getHTTPTimeoutPolicy(void) const { return fnPtrResponder_timeout; }
private:
void (*mCallback)(void **,S32);
@@ -552,11 +557,11 @@ BOOL LLMessageSystem::checkMessages( S64 frame_count )
S32 true_rcv_size = 0;
U8* buffer = mTrueReceiveBuffer;
mTrueReceiveSize = mPacketRing->receivePacket(mSocket, (char *)mTrueReceiveBuffer);
// If you want to dump all received packets into SecondLife.log, uncomment this
//dumpPacketToLog();
receive_size = mTrueReceiveSize;
mLastSender = mPacketRing->getLastSender();
mLastReceivingIF = mPacketRing->getLastReceivingInterface();
@@ -1826,6 +1831,10 @@ void process_start_ping_check(LLMessageSystem *msgsystem, void** /*user_data*/)
// Note: this is currently unused. --mark
void open_circuit(LLMessageSystem *msgsystem, void** /*user_data*/)
{
llassert_always(false);
return;
#if 0
U32 ip;
U16 port;
@@ -1834,6 +1843,7 @@ void open_circuit(LLMessageSystem *msgsystem, void** /*user_data*/)
// By default, OpenCircuit's are untrusted
msgsystem->enableCircuit(LLHost(ip, port), FALSE);
#endif
}
void close_circuit(LLMessageSystem *msgsystem, void** /*user_data*/)

4
indra/llmessage/message.h
View File

@@ -233,6 +233,8 @@ class LLMessageSystem : public LLMessageSenderInterface
private:
message_template_name_map_t mMessageTemplates;
message_template_number_map_t mMessageNumbers;
friend class LLFloaterMessageLogItem;
friend class LLFloaterMessageLog;
public:
S32 mSystemVersionMajor;
@@ -341,7 +343,7 @@ public:
bool addCircuitCode(U32 code, const LLUUID& session_id);
BOOL poll(F32 seconds); // Number of seconds that we want to block waiting for data, returns if data was received
BOOL checkMessages( S64 frame_count = 0 );
BOOL checkMessages(S64 frame_count = 0);
void processAcks();
BOOL isMessageFast(const char *msg);

7
indra/llplugin/CMakeLists.txt
View File

@@ -3,23 +3,16 @@
project(llplugin)
include(00-Common)
include(CURL)
include(LLCommon)
include(LLImage)
include(LLMath)
include(LLMessage)
include(LLRender)
include(LLXML)
include(LLWindow)
include_directories(
${LLCOMMON_INCLUDE_DIRS}
${LLIMAGE_INCLUDE_DIRS}
${LLMATH_INCLUDE_DIRS}
${LLMESSAGE_INCLUDE_DIRS}
${LLRENDER_INCLUDE_DIRS}
${LLXML_INCLUDE_DIRS}
${LLWINDOW_INCLUDE_DIRS}
${LLQTWEBKIT_INCLUDE_DIR}
)

2
indra/llplugin/llpluginclassmedia.cpp
View File

@@ -39,8 +39,6 @@
#include "llpluginclassmedia.h"
#include "llpluginmessageclasses.h"
#include "llqtwebkit.h"
static int LOW_PRIORITY_TEXTURE_SIZE_DEFAULT = 256;
static int nextPowerOf2( int value )

11
indra/llplugin/slplugin/slplugin.cpp
View File

@@ -124,14 +124,9 @@ BOOL PreventSetUnhandledExceptionFilter()
newJump[ 0 ] = 0xE9; // JMP absolute
memcpy( &newJump[ 1 ], &dwRelativeAddr, sizeof( pNewFunc ) );
//SIZE_T bytesWritten;
//BOOL bRet = WriteProcessMemory( GetCurrentProcess(), pOrgEntry, newJump, sizeof( pNewFunc ) + 1, &bytesWritten );
DWORD oldProtect;
BOOL bRet = VirtualProtect(pOrgEntry, sizeof(pNewFunc) + 1, PAGE_READWRITE, &oldProtect);
if (!bRet) return FALSE;
memcpy(pOrgEntry, newJump, sizeof(pNewFunc) + 1);
VirtualProtect(pOrgEntry, sizeof(pNewFunc) + 1, oldProtect, &oldProtect);
return TRUE;
SIZE_T bytesWritten;
BOOL bRet = WriteProcessMemory( GetCurrentProcess(), pOrgEntry, newJump, sizeof( pNewFunc ) + 1, &bytesWritten );
return bRet;
#else
return FALSE;
#endif

70
indra/llqtwebkit/CMakeLists.txt Normal file
View File

@@ -0,0 +1,70 @@
# -*- cmake -*-
project(llqtwebkit)
include(00-Common)
include(Qt4)
if(NOT WORD_SIZE EQUAL 32)
if(WINDOWS)
add_definitions(/FIXED:NO)
else(WINDOWS)
add_definitions(-fPIC)
endif(WINDOWS)
endif(NOT WORD_SIZE EQUAL 32)
include_directories(${QT_INCLUDES})
add_subdirectory(qtwebkit_cookiejar)
include_directories(qtwebkit_cookiejar/src/)
set(llqtwebkit_SOURCE_FILES
llembeddedbrowser.cpp
llembeddedbrowserwindow.cpp
lljsobject.cpp
llnetworkaccessmanager.cpp
llqtwebkit.cpp
llstyle.cpp
llwebpage.cpp
llwebpageopenshim.cpp
)
set(llqtwebkit_HEADER_FILES
llembeddedbrowser.h
llembeddedbrowser_p.h
llembeddedbrowserwindow.h
llembeddedbrowserwindow_p.h
lljsobject.h
llnetworkaccessmanager.h
llqtwebkit.h
llstyle.h
llwebpage.h
llwebpageopenshim.h
pstdint.h
)
set(llqtwebkit_UI_FILES
passworddialog.ui
)
set(llqtwebkit_LINK_LIBRARIES
networkcookiejar
)
QT4_WRAP_UI(llqtwebkit_UI_MOC ${llqtwebkit_UI_FILES})
QT4_WRAP_CPP(llqtwebkit_HEADERS_MOC ${llqtwebkit_HEADER_FILES})
include_directories(${CMAKE_CURRENT_BINARY_DIR})
add_library(llqtwebkit
${llqtwebkit_SOURCE_FILES}
${llqtwebkit_HEADERS_MOC}
${llqtwebkit_UI_MOC}
)
add_dependencies(llqtwebkit prepare)
target_link_libraries(llqtwebkit ${llqtwebkit_LINK_LIBRARIES})
add_dependencies(llqtwebkit
networkcookiejar
)

14
indra/llqtwebkit/autotests/llembeddedbrowser/llembeddedbrowser.pro Normal file
View File

@@ -0,0 +1,14 @@
TEMPLATE = app
TARGET =
DEPENDPATH += .
INCLUDEPATH += .
CONFIG += qtestlib
QT += webkit opengl network
include(../../llmozlib2.pri)
DEFINES += AUTOTEST
# Input
SOURCES += tst_llembeddedbrowser.cpp

Some files were not shown because too many files have changed in this diff Show More