6 EVL Functions
As an ‘EVL value’ in anon-config file, arbitrary EVL functions and expressions can be used.
An input field is represented as ‘IN’.
All functions can be used in two ways:
- with pointers (preferred)
- without pointers (i.e. as referenced values, “with star”)
Option with pointers is preferred as it can handle NULL values (‘nullptr’ in fact).
So these two examples:
str_function(IN)
str_function(*IN)
are basically the same, but the first one might fail in case of using some standard C++ function and NULL value arrive. All EVL functions handle NULLs and (mostly) returns also NULL, so use ‘IN’ for them. In all other cases use better ‘*IN’.
There are these two rules in all EVL string manipulation functions described in this section:
- When the first argument is a pointer, the function returns also a pointer.
- When the first argument is ‘
nullptr’, the function returns ‘nullptr’ as well.
6.1 Randomization Functions
For randomization functions are used same rules regarding ‘nullptr’ as for string functions.
randomize()
(since EVL 2.1)
Examples:
// random int from whole int range
out->random_int = randomize(in->value);
// random int from interval < value - 1000 , value + 2000 >
out->random_int_range = randomize(in->value,-1000,2000);
random_int()
random_long()
random_short()
random_char()
(since EVL 2.1)
Examples:
// random value from whole int range
out->random_value = random_int();
// random value from interval <1000,2000>
out->random_range = random_int(1000,2000);
random_float()
random_double()
(since EVL 2.1)
Examples:
// random value from whole float range
out->random_value = random_float();
// random float value from interval <1000,2000>
out->random_range = random_float(1000,2000);
random_decimal()
(since EVL 2.1)
Examples:
// random value from whole decimal range
out->random_value = random_decimal();
// random float value from interval <1000,2000>
out->random_range = random_decimal(1000,2000);
random_date()
random_datetime()
random_timestamp()
(since EVL 2.1)
Examples:
// random date between 1970-01-01 and 2069-12-31
out->random_value = random_date();
// random date from this century
out->random_range = random_date(date("2000-01-01"), date("2099-12-31"));
random_string()
(since EVL 2.1)
Examples:
// random string of length between 0 and 10
out->random_value = random_string();
// random string of length 5
out->random_range = random_string(5,5);
6.2 String Functions
All string manipulation functions can be used in two ways:
- with pointers (preferred)
- without pointers (i.e. as referenced values, “with star”)
Option with pointers is preferred as it can handle NULL values (‘nullptr’ in fact).
So these two examples:
out->field = str_function(in->field);
*out->field = str_function(*in->field);
are basically the same, but the second one will fail in case ‘in->field’ will be NULL (i.e. ‘nullptr’).
There are these two rules in all string manipulation functions described in this section:
- When the first argument is a pointer, the function returns also a pointer.
- When the first argument is ‘
nullptr’, the function returns ‘nullptr’ as well.
6.2.1 length
(since EVL 2.0)
Returns the length of given string.
For ‘nullptr’ it returns again ‘nullptr’.
Example:
length((string)"Some text") // return 9
length(nullptr) // return nullptr
In mapping it might look like this (without pointers):
out->str_len = length(in->first_name);
6.2.2 split
(since EVL 1.3)
Example:
split("Some text, another text.", ' ')
// returns vector ["Some", "text,", "another", "text."]
When the first argument is ‘nullptr’, it returns ‘nullptr’.
In mapping it might look like this (without pointers):
static std::vector<std::string> name_vec;
name_vec = split(*in->full_name", ' ');
*out->first_name = name_vec[0];
*out->last_name = name_vec[1];
or (preferably) using pointers:
static std::vector<std::string*>* name_vec;
name_vec = split(in->full_name", ' ');
out->first_name = name_vec[0];
out->last_name = name_vec[1];
Function headers:
std::vector<std::string> split(const std::string& str, \
const char delimiter);
std::vector<std::string*>* split(const std::string* const str, \
const char delimiter);
6.2.3 starts_with, ends_with
(since EVL 2.0)
True if a string starts or ends with the given substring.
When the first argument is ‘nullptr’, it returns False.
Example:
starts_with("Some text", "Some") // return True
starts_with("Some text", "x") // return False
starts_with(nullptr, "x") // return False
ends_with("Some text", "ext") // return True
ends_with("Some text", "x") // return False
In mapping it might look like this:
*out->test_field = starts_with(in->test_field ? "OK" : "NOK" ;
Function headers:
bool starts_with(const std::string& str, const char* const prefix);
bool starts_with(const std::string* const str, const char* const prefix);
bool starts_with(const std::string& str, const std::string& prefix);
bool starts_with(const std::string* const str, const std::string& prefix);
bool ends_with(const std::string& str, const char* const suffix);
bool ends_with(const std::string* const str, const char* const suffix);
bool ends_with(const std::string& str, const std::string& suffix);
bool ends_with(const std::string* const str, const std::string& suffix);
6.2.4 str_compress, str_uncompress
(since EVL 2.0)
Compress/uncompress the given string. Examples which return pointers:
str_compress(in->string_field_to_compress) // snappy by default
str_compress(in->string_field_to_compress, compression::gzip)
str_compress(in->snappy_field) // snappy by default
str_compress(in->gzipped_field, compression::gzip)
Examples which return string values:
str_compress(*in->string_field_to_compress) // snappy by default
str_compress(*in->string_field_to_compress, compression::gzip)
str_compress(*in->snappy_field) // snappy by default
str_compress(*in->gzipped_field, compression::gzip)
When the first argument is ‘nullptr’, it returns ‘nullptr’.
In mapping it might look like this:
out->gzipped_field = str_compress(in->string_field);
Function headers:
std::string str_compress(const std::string& str, \
const compression method = compression::snappy);
std::string* str_compress(const std::string* const str, \
const compression method = compression::snappy);
std::string str_uncompress(const std::string& str, \
const compression method = compression::snappy);
std::string* str_uncompress(const std::string* const str, \
const compression method = compression::snappy);
6.2.5 str_count
(since EVL 1.3)
It counts the number of occurrences of given string or character. Example:
str_count("Some text, another text.", ' ') // returns 3
str_count("Some text, another text.", "text") // returns 2
When the first argument is ‘nullptr’, it returns ‘nullptr’.
In mapping it might look like this (using pointers):
out->jan_cnt = str_count(in->first_name", "Jan");
or without pointers:
*out->jan_cnt = str_count(*in->first_name", "Jan");
Function headers:
std::size_t str_count(const std::string& str, const char ch);
std::size_t* str_count(const std::string* const str, const char ch);
std::size_t str_count(const std::string& str, const char* const substr);
std::size_t* str_count(const std::string* const str, \
const char* const substr);
std::size_t str_count(const std::string& str, const std::string& substr);
std::size_t* str_count(const std::string* const str, \
const std::string& substr);
6.2.6 str_index, str_rindex
(since EVL 2.0)
str_index(str,substr)
it returns the index (counted from 0) of the first occurrence of the given substring,
str_rindex(str,substr)
it returns the index (counted from 0) of the last occurrence of the given substring.
When no match, then ‘-1’ is returned.
When the string is ‘nullptr’, it returns ‘nullptr’.
Examples:
str_index("Some text text", "text") // return 5
str_index("Some text text", "xyz") // return -1
str_index(nullptr, 'x') // return nullptr
str_rindex("Some text text", "text") // return 10
Function headers:
std::int64_t str_index(const std::string& str, const char* const substr);
std::int64_t* str_index(const std::string* const str, \
const char* const substr);
std::int64_t str_index(const std::string& str, const std::string& substr);
std::int64_t* str_index(const std::string* const str, \
const std::string& substr);
std::int64_t str_rindex(const std::string& str, const char* const substr);
std::int64_t* str_rindex(const std::string* const str, \
const char* const substr);
std::int64_t str_rindex(const std::string& str, const std::string& substr);
std::int64_t* str_rindex(const std::string* const str, \
const std::string& substr);
6.2.7 str_join
(since EVL 2.4)
str_join(vector_of_strings,delimiter)
it returns the string of concatenated vector members, delimited by a specified delimiter.
When the vector is ‘nullptr’, it returns ‘nullptr’.
Examples of a mapping:
static std::vector<std::string> x{"Here", "is", "a", "hardcoded", "vector."};
*out->x_spaced = str_join(x,' ') // return "Here is a hardcoded vector."
*out->x_dashed = str_join(x,'-') // return "Here-is-a-hardcoded-vector."
*out->x_longer = str_join(x,"---") // return "Here---is---a---hardcoded---vector."
Function headers:
std::string str_join(const std::vector<std::string>& strings, \
const char delimiter);
std::string* str_join(const std::vector<std::string*>* strings, \
const char delimiter);
std::string str_join(const std::vector<std::string>& strings, \
const std::string_view delimiter);
std::string* str_join(const std::vector<std::string*>* strings, \
const std::string_view delimiter);
6.2.8 str_mask_left, str_mask_right
(since EVL 2.1)
Functions return string with visible characters replaced by given character from given direction, but keep the specified number of character unchanged.
Example:
str_mask_left("abcd text efgh", 6) // returns "abcd tex* ****"
str_mask_right("1234567890", 3, '-') // returns "---4567890"
Without the second argument, asterisk ‘*’ is assumed.
When the first argument is ‘nullptr’, these functions return ‘nullptr’.
Function headers:
std::string str_mask_left(const std::string& str, \
const std::size_t keep, const char ch = '*');
std::string* str_mask_left(const std::string* const str, \
const std::size_t keep, const char ch = '*');
std::string str_mask_right(const std::string& str, \
const std::size_t keep, const char ch = '*');
std::string* str_mask_right(const std::string* const str, \
const std::size_t keep, const char ch = '*');
6.2.9 str_pad_left, str_pad_right
(since EVL 2.1)
Add from left/right the specified character (space by default), up to the given length. It counts Bytes, not characters, so be careful with multibyte encodings.
Example:
str_pad_left("123",7,'0') // returns "0000123"
str_pad_right("text",7) // returns "text "
str_pad_right("text",2) // returns "text"
str_pad_left("Groß",6,'*') // returns "*Groß" as "ß" has 2 Bytes
When the first argument is ‘nullptr’, these functions return ‘nullptr’.
Function headers:
std::string str_pad_left(const std::string& str, \
const std::size_t length, const char ch = ' ');
std::string* str_pad_left(const std::string* const str, \
const std::size_t length, const char ch = ' ');
std::string str_pad_right(const std::string& str, \
const std::size_t length, const char ch = ' ');
std::string* str_pad_right(const std::string* const str, \
const std::size_t length, const char ch = ' ');
6.2.10 str_replace
(since EVL 1.3)
Examples:
str_replace("Some text", ' ', '-') // returns "Some-text"
str_replace("Some text", "Some", "Any") // returns "Any text"
str_replace("Some text", ' ', "SPACE") // returns "SomeSPACEtext"
When the first argument is ‘nullptr’, it returns ‘nullptr’.
In mapping it might look like this:
out->name = str_replace(in->name", ' ', '-');
Function headers:
std::string str_replace(const std::string& str, \
const char old_ch, const char new_ch);
std::string* str_replace(const std::string* const str, \
const char old_ch, const char new_ch);
std::string str_replace(const std::string& str, \
const char* const old_substr, const char* const new_substr);
std::string* str_replace(const std::string* const str, \
const char* const old_substr, const char* const new_substr);
std::string str_replace(const std::string& str, \
const std::string& old_substr, const std::string& new_substr);
std::string* str_replace(const std::string* const str, \
const std::string& old_substr, const std::string& new_substr);
6.2.11 str_to_base64, base64_to_str
(since EVL 2.6)
Encode/decode string to/from Base64 form.
When the first argument is ‘nullptr’, it returns also ‘nullptr’.
Examples:
str_to_base64("Some\r\nbíňářý text.") // return "U29tZQ0KYsOtxYjDocWZw70gdGV4dC4="
base64_to_str("U29tZQ0KYsOtxYjDocWZw70gdGV4dC4=") // return "Some\r\nbíňářý text."
Function headers:
std::string str_to_base64(const std::string& str);
std::string* str_to_base64(const std::string* const str);
std::string base64_to_str(const std::string& str);
std::string* base64_to_str(const std::string* const str);
6.2.12 str_to_hex, hex_to_str
(since EVL 2.0)
Convert string or ustring to its hexadecimal representation and vice versa. (Ustring support has been added in EVL v2.6.)
When the first argument is ‘nullptr’, it returns also ‘nullptr’.
Examples:
str_to_hex("Some text") // return "536f6d652074657874"
hex_to_str("536f6d652074657874") // return "Some text"
Function headers:
std::string str_to_hex(const std::string& str);
std::string* str_to_hex(const std::string* const str);
ustring str_to_hex(const __detail::u16str& str);
ustring* str_to_hex(const ustring* const str);
std::string hex_to_str(const std::string& str);
std::string* hex_to_str(const std::string* const str);
ustring hex_to_str(const __detail::u16str& str);
ustring* hex_to_str(const ustring* const str);
6.2.13 substr
(since EVL 2.0)
Return a substring starting after given position with the specified length.
Example:
substr("123456789",0,2) // returns "12"
substr("123456789",6) // returns "789"
Without the third argument, it returns the rest of the string.
When the first argument is ‘nullptr’, function returns ‘nullptr’.
Function headers:
std::string substr(const std::string& str, const std::size_t pos = 0,
const std::int64_t count = std::numeric_limits<std::int64_t>::max());
std::string* substr(const std::string* const str, const std::size_t pos = 0,
const std::int64_t count = std::numeric_limits<std::int64_t>::max());
6.2.14 trim, trim_left, trim_right
(since EVL 1.0)
Example:
trim(" text ") // returns "text"
trim_left(" text ") // returns "text "
trim_right("--text---", '-') // returns "--text"
Trim character ‘char’ from both sides, from left, from right, respectively.
Without the second argument, space is assumed.
When the first argument is ‘nullptr’, these functions return ‘nullptr’.
Function headers:
std::string trim(const std::string& str, const char ch = ' ');
std::string* trim(const std::string* const str, const char ch = ' ');
std::string trim_left(const std::string& str, const char ch = ' ');
std::string* trim_left(const std::string* const str, const char ch = ' ');
std::string trim_right(const std::string& str, const char ch = ' ');
std::string* trim_right(const std::string* const str, const char ch = ' ');
6.2.15 uppercase, lowercase
(since EVL 1.0)
Examples:
uppercase("AbCd") // returns "ABCD"
lowercase("AbCd") // returns "abcd"
When the argument is ‘nullptr’, these functions return ‘nullptr’.
Without specifying the second parameter it acts only on ‘A-Z’ and ‘a-z’.
When there is a need to acts also on national letters (with diacritics for example), there can be the second parameter specified with the locale:
static std::locale de_locale("de_DE.utf8");
*out->field_upcase = uppercase(*in->field, de_locale);
It is possible to specify the locale in the function as string, but using the static specification of locale is recommended due to performance.
Function headers:
std::string uppercase(const std::string& str);
std::string* uppercase(const std::string* const str);
std::string uppercase(const std::string& str, const std::locale& locale);
std::string* uppercase(const std::string* const str, const std::locale& locale);
std::string lowercase(const std::string& str);
std::string* lowercase(const std::string* const str);
std::string lowercase(const std::string& str, const std::locale& locale);
std::string* lowercase(const std::string* const str, const std::locale& locale);
6.3 Checksum Functions
md5sum(str)
sha224sum(str)
sha256sum(str)
sha384sum(str)
sha512sum(str)
(since EVL 1.0)
these standard checksum functions can be used in mapping this way for example:
*out->anonymized_username = sha256sum(*in->username);
When the argument is ‘nullptr’, it returns ‘nullptr’.
But in such case you need to use pointer manipulation, so the example would look like:
out->anonymized_username = sha256sum(in->username);
Functions headers:
std::string md5sum(const char* const str);
std::string md5sum(const std::string& str);
std::string* md5sum(const std::string* const str);
std::string sha224sum(const char* const str);
std::string sha224sum(const std::string& str);
std::string* sha224sum(const std::string* const str);
std::string sha256sum(const char* const str);
std::string sha256sum(const std::string& str);
std::string* sha256sum(const std::string* const str);
std::string sha384sum(const char* const str);
std::string sha384sum(const std::string& str);
std::string* sha384sum(const std::string* const str);
std::string sha512sum(const char* const str);
std::string sha512sum(const std::string& str);
std::string* sha512sum(const std::string* const str);
6.4 IP Addresses Functions
Typical IPv4 manipulation usage within a mapping:
// convert and assign IPv4 string into unsigned integer
out->ipv4_uint = str_to_ipv4(in->ipv4_string);
// or the other way
out->ipv4_string = ipv4_to_str(in->ipv4_uint);
Typical IPv6 manipulation usage within a mapping:
// suppose in->ipv6_string = "4567::123"
out->ipv6_normalized = ipv6_normalize(in->ipv6_string);
// return "4567:0000:0000:0000:0000:0000:0000:0123"
// suppose in->ipv6_string = "0000:0000:0000:0004:5678:9098:0000:0654"
out->ipv6_compressed = ipv6_compress(in->ipv6_string);
// return "::4:5678:9098:0000:654"
Or one can distinguish both IP versions:
if ( is_valid_ipv4(in->ip_string) ) {
// act on IPv4
}
else if ( is_valid_ipv6(in->ip_string) ) {
// act on IPv6
}
else {
// act when neither is valid
}
There are these two rules in all IP manipulation functions described in this section:
- When the first argument is a pointer, the function returns also a pointer.
- When the first argument is ‘
nullptr’, the function returns ‘nullptr’ as well.
6.4.1 IPv4 Functions
(since EVL 2.4)
‘ipv4addr’
constructor
‘str_to_ipv4()’
convert string to uint32,
‘ipv4_to_str()’
convert uint32 to ipv4 string,
‘is_valid_ipv4()’
to check whether the string is valid IPv4.
6.4.2 IPv6 Functions
(since EVL 2.4)
‘str_to_ipv6()’
convert string to uint128,
‘ipv6_to_str()’
convert uint128 to ipv6 string,
‘is_valid_ipv6()’
to check whether the string is valid IPv6,
‘ipv6_normalize()’
convert string to normalized IPv6 string,
‘ipv6_compress()’
convert string to compressed IPv6 string,
Examples
To get normalized and compressed IPv6:
// suppose in->ipv6_string = "0000:0000:22::0003:4"
out->ipv6_normalized = ipv6_normalize(in->ipv6_string);
// "0000:0000:0022:0000:0000:0000:0003:0004"
out->ipv6_compressed = ipv6_compress(in->ipv6_string);
// "0:0:22::3:4"