Commit Graph

3787 Commits

Author SHA1 Message Date
Thierry FOURNIER
6677959879 DOC: pattern: pattern organisation schematics 2014-03-20 14:10:49 +01:00
Willy Tarreau
0e9b1b4d1f MEDIUM: compression: consider the "q=" attribute in Accept-Encoding
Till now we didn't consider "q=". It's problematic because the first
effect is that compression tokens were not even matched if it was
present.

It is important to parse it correctly because we still want to allow
a user-agent to send "q=0" to explicitly disable a compressor, or to
specify its preferences.

Now, q-values are respected in order of precedence, and when several
q-values are equal, the first occurrence is used.
2014-03-19 12:12:01 +01:00
Julien Vehent
f21be328e8 DOC: lowercase format string in unique-id
Update configuration.txt to use the correct format string in the
unique-id documentation.
2014-03-18 08:18:38 +01:00
Thierry FOURNIER
e5978bfc25 BUG/MEDIUM: acl: boolean only matches were broken by recent changes
The ACL changes made in the last patchset force the execution
of each pattern matching function. The function pat_match_nothing
was not provided to be excuted, it was just used as a flag that
was checked by the ACL execution code. Now this function is
executed and always returns false.

This patch makes it work as expected. Now, it returns the boolean
status of the received sample just as was done previously in the
ACL code.

This bug is a part of the patchset just merged. It does not need
to be backported.
2014-03-17 20:12:40 +01:00
Thierry FOURNIER
c5a4e98639 MEDIUM: acl: Change the acl register struct
This patch replace a lot of pointeur by pattern matching identifier. If
the declared ACL use all the predefined pattern matching functions, the
register function gets the functions provided by "pattern.c" and
identified by the PAT_LATCH_*.

In the case of the acl uses his own functions, they can be declared, and
the acl registration doesn't change it.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
5b16df74b8 DOC: document "get map" / "get acl" on the CLI 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
b7729c96a4 MINOR: pattern: forbid dns resolutions
This patch adds the flags "-n" on the acl parser. the flag "-n" forbif
the DNS resolutions. The maps have always the dns resolutions disabled.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
66eb9bf691 MINOR: pattern: Remove the flag "PAT_F_FROM_FILE".
This flag is no longer used. The last place using this, are the display
of the result of pattern matching in the cli command "get map" or "get
acl".

The first parameter of this command is the reference of the file used to
perform the lookup.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
fc7ac7b89c MINOR: standard: Disable ip resolution during the runtime
The function str2net runs DNS resolution if valid ip cannot be parsed.
The DNS function used is the standard function of the libc and it
performs asynchronous request.

The asynchronous request is not compatible with the haproxy
archictecture.

str2net() is used during the runtime throught the "socket".

This patch remove the DNS resolution during the runtime.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
94580c9f52 MINOR: dumpstat/conf: display all the configuration lines that using pattern reference 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
eeaa951726 MINOR: configuration: File and line propagation
This patch permits to communicate file and line of the
configuration file at the configuration parser.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
31db4ae7b4 MINOR: pattern: index duplicates
The indexation functions now accept duplicates. This way it is possible
to always have some consistency between lists and trees. The "add" command
will always add regardless of any previous existence. The new entry will
not be used because both trees and list retrieve keys in insertion order.
Thus the "add" operation will always succeed (as long as there is enough
memory).
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
d32079e30e MINOR: doc: Update the documentation about the map and acl
Documentation about the socket command line new commands and new behavior.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
64c585ffeb MINOR: cli: Block the usage of the command "acl add" in many cases.
If acl is shared with a map, the "add acl" command must be blocked
because it not take a sample on his parameters. The absense of this
parameter can cause error with corresponding maps.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
0b6d15fdc8 MINOR: regex: The pointer regstr in the struc regex is no longer used.
The pointer <regstr> is only used to compare and identify the original
regex string with the patterns. Now the patterns have a reference map
containing this original string. It is useless to store this value two
times.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
149e0fe44f MINOR: pattern: The function pat_ref_set() have now atomic behavior
Before this patch, this function try to add values in best effort. If
the parsing iof the value fail, the operation continue until the end.

Now, this function stop on the first error and left the pattern in
coherant state.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
364cfdff7a MEDIUM: dumpstats: Display error message during add of values.
This patch adds new display type. This display returns allocated string,
when the string is flush into buffers, it is freed. This permit to
return the content of "memprintf(err, ...)" messages.

The pat_ref_add functions has changed to return error.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
9860c41258 MINOR: acl/pattern: Acl "-M" option force to load file as map file with two columns 2014-03-17 18:06:08 +01:00
Thierry FOURNIER
c0bd9100eb MINOR: pattern: Check if the file reference is not used with acl and map
The format of the acl file are not the same than the format of the map
files. In some case, the same file can be used, but this is ambiguous
for the user because the patterns are not the expected.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
39bef456fb MINOR: pattern/map/acl: Centralization of the file parsers
The acl and map function do the same work with the file parsing. This
patch merge these code in only one.

Note that the function map_read_entries_from_file() in the file "map.c"
is moved to the the function pat_ref_read_from_file_smp() in the file
"pattern.c". The code of this function is not modified, only the the
name and the arguments order has changed.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
9356c68727 MEDIUM: dumpstats/pattern: display and use each pointer of each pattern dumped
Each pattern displayed is associated to the value of his pattern
reference. This value can be used for deleting the entry. It is useful
with complex regex: the users are not forced to write the regex with all
the amiguous chars and escaped chars on the CLI.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
e369ca2e66 MEDIUM: pattern_find_smp: functions find_smp uses the pat_ref_elt to find the element to be removed
The find_smp search the smp using the value of the pat_ref_elt pointer.

The pat_find_smp_* are no longer used. The function pattern_find_smp()
known all pattern indexation, and can be found
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
7acca4b269 MEDIUM: pattern: delete() function uses the pat_ref_elt to find the element to be removed
All the pattern delete function can use her reference to the original
"struct pat_ref_elt" to find the element to be remove. The functions
pat_del_list_str() and pat_del_meth() were deleted because after
applying this modification, they have the same code than pat_del_list_ptr().
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
86db66a7c8 MINOR: dumpstats: display the reference for th key/pattern and value.
Before this patch, the "get map/acl" function try to convert and display
the sample. This behavior is not efficient because some type like the
regex cannot be reversed and displayed as string.

This patch display the original stored reference.
2014-03-17 18:06:08 +01:00
Thierry FOURNIER
6bb53ff164 MINOR: pattern: Each pattern expression element store the reference struct.
Now, each pattern entry known the original "struct pat_ref_elt" from
that was built. This patch permit to delete each pattern entry without
confusion. After this patch, each reference can use his pointer to be
targeted.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
0d6ba513a5 MINOR: pattern: store configuration reference for each acl or map pattern.
This patch permit to add reference for each pattern reference. This is
useful to identify the acl listed.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d25c842326 MINOR: pattern: Merge function pattern_add() with pat_ref_push().
The function Pattern_add() is only used by pat_ref_push(). This patch
remove the function pattern_add() and merge his code in the function
pat_ref_push().
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
aa222aadb6 MINOR: pattern: The function "pattern_register()" is no longer used.
Remove the function "pattern_register()" and its prototype because it
is no longer used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
3534d881bc MINOR: pattern/acl: Each pattern of each acl can be load with specified id
This patch adds -u option on the acl pattern loading to force a unique ID
value.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
af5a29d5f8 MINOR: pattern: Each pattern is identified by unique id.
The pattern reference are stored with two identifiers: the unique_id and
the reference.

The reference identify a file. Each file with the same name point to the
same reference. We can register many times one file. If the file is
modified, all his dependencies are also modified. The reference can be
used with map or acl.

The unique_id identify inline acl. The unique id is unique for each acl.
You cannot force the same id in the configuration file, because this
repport an error.

The format of the acl and map listing through the "socket" has changed
for displaying these new ids.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5d34408785 MEDIUM: pattern: The expected type is stored in the pattern head, and conversion is executed once.
This patch extract the expect_type variable from the "struct pattern" to
"struct pattern_head". This variable is set during the declaration of
ACL and MAP. With this change, the function "pat_parse_len()" become
useless and can be replaced by "pat_parse_int()".

Implicit ACLs by default rely on the fetch's output type, so let's simply do
the same for all other ones. It has been verified that they all match.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
c5959fd5d4 MEDIUM: pattern: merge same pattern
Sometimes the same pattern file is used with the same index, parse and
parse_smp functions. If this two condition are true, these two pattern
are identical and the same struct can be used.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1e00d3853b MAJOR: pattern/map: Extends the map edition system in the patterns
This patch add the following socket command line options:

  show acl [<id>]
  clear acl <id>
  get acl <id> <pattern>
  del acl <id> <pattern>
  add acl <id> <pattern>

The system used for maps is backported in the pattern functions.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
888863534c MINOR: map/pattern: The sample parser is stored in the pattern
We cannot separe the pattern and the value. Now, the patern known the
value and the pattern is able to parsehis associated sample staroage.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
3667e514d9 MEDIUM: pattern/dumpstats: The function pattern_lookup() is no longer used
This function are used in dumpstats. Now this function is replaced by
delete and find_smp function pointer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
55d0b10f06 MEDIUM: pattern: add sample lookup function.
Some functions needs to change the sample associated to pattern. This
new pointer permit to return the a pointer to the sample pointer. The
caller can use or change the value.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
6f7203d673 MEDIUM: pattern: add prune function
This path add specific pointer to each expression to point on prune
function. Now, each pattern expression embed his own prune function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b113650e54 MEDIUM: pattern: add delete functions
This commit adds a delete function for patterns. It looks up all
instances of the pattern to delete and deletes them all. The fetch
keyword declarations have been extended to point to the appropriate
delete function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
33a7433ac9 MEDIUM: pattern: Index IPv6 addresses in a tree.
This commit adds second tree node in the pattern struct and use it to
index IPv6 addresses. This commit report feature used in the list. If
IPv4 not match the tree, try to convert the IPv4 address in IPv6 with
prefixing the IPv4 address by "::ffff", after this operation, the match
function try lookup in the IPv6 tree. If the IPv6 sample dont match the
IPv6 tree, try to convert the IPv6 addresses prefixed by "2002:IPv4",
"::ffff:IPv4" and "::0000:IPv4" in IPv4 address. after this operation,
the match function try lookup in the IPv4 tree.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
5338eea8eb MEDIUM: pattern: The match function browse itself the list or the tree.
The match function known the format of the pattern. The pattern can be
stored in a list or in a tree. The pattern matching function use itself
the good entry point and indexation type.

Each pattern matching function return the struct pattern that match. If
the flag "fill" is set, the struct pattern is filled, otherwise the
content of this struct must not be used.

With this feature, the general pattern matching function cannot have
exceptions for building the "struct pattern".
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b4be12952a MINOR: map/dumpstats: The cli cmd "get map ..." display the "int" format.
The "get map ..." command display the integer range that match the
request.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b990384c9a MINOR: dumpstats: change the "get map" display management
The original get map display function set the comma separator after each
word displayed. This is not efficient because we cannot knew if the
displayed word is the last.

This new system set the comma separator before the displayed word, and
independant "\n" is set a the end of the function.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d572343d51 MINOR: dumpstats: change map inline sentences
The errors and messages used in the cli are sometimes impossibkle to
understand. This patch changes all these messages.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1794fdf37e MEDIUM: pattern: The function pattern_exec_match() returns "struct pattern" if the patten match.
Before this commit, the pattern_exec_match() function returns the
associate sample, the associate struct pattern or the associate struct
pattern_tree. This is complex to use, because we can check the type of
information returned.

Now the function return always a "struct pattern". If <fill> is not set,
only the value of the pointer can be used as boolean (NULL or other). If
<fill> is set, you can use the <smp> pointer and the pattern
information.

If information must be duplicated, it is stored in trash buffer.
Otherwise, the pattern can point on existing strings.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
1432a0c5db MINOR: dumpstats: Group map inline help 2014-03-17 18:06:07 +01:00
Thierry FOURNIER
d437314979 MEDIUM: sample/http_proto: Add new type called method
The method are actuelly stored using two types. Integer if the method is
known and string if the method is not known. The fetch is declared as
UINT, but in some case it can provides STR.

This patch create new type called METH. This type contain interge for
known method and string for the other methods. It can be used with
automatic converters.

The pattern matching can expect method.

During the free or prune function, http_meth pettern is freed. This
patch initialise the freed pointer to NULL.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
7654c9ff44 MEDIUM: sample: Remove types SMP_T_CSTR and SMP_T_CBIN, replace it by SMP_F_CONST flags
The operations applied on types SMP_T_CSTR and SMP_T_STR are the same,
but the check code and the declarations are double, because it must
declare action for SMP_T_C* and SMP_T_*. The declared actions and checks
are the same. this complexify the code. Only the "conv" functions can
change from "C*" to "*"

Now, if a function needs to modify input string, it can call the new
function smp_dup(). This one duplicate data in a trash buffer.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
b050463375 MINOR: standard: Add function for converting cidr to network mask. 2014-03-17 18:06:07 +01:00
Thierry FOURNIER
0e9af55700 MINOR: sample: dont call the sample cast function "c_none"
If the cast function to execute is c_none, dont execute it and return
true. The function c_none, do nothing. This save a call.
2014-03-17 18:06:07 +01:00
Thierry FOURNIER
e1bcac5b8f MINOR: pattern: Rename "pat_idx_elt" to "pattern_tree"
This is just for having coherent struct names.
2014-03-17 18:06:07 +01:00