Mapping-Components

9 Mapping Components ¶

These components perform a mapping specified by an evm file.

• Aggreg – do aggregations on the groups of records specified by a key,
• Join – join two inputs by a key,
• Map – generic mapping component, can be used also for pivoting for example.

---

9.1 Aggreg ¶

(since EVL 1.0)

Applies aggregation mapping on each group of records based on the <key>.

Aggreg
is to be used in EVS job structure definition file. <f_in> and <f_out> are either input and output file or flow name.

evl aggreg
is intended for standalone usage, i.e. to be invoked from command line and reading records from standard input and writing to standard output.

EVD, EVM and EVS are EVL definition files, for details see evl-evd(5), evl-evm(5) and evl-evs(5).

Synopsis ¶

Aggreg
  <f_in> <f_out> (<evd_in>|-D <inline_evd>) (<evd_out>|-d <inline_evd>)
  <evm> --key=<key>
  [-c|--check-sort] [-i|--ignore-case] [-x|--text-input] [-y|--text-output]
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [--reject <f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]

evl aggreg
  (<evd_in> | -D <inline_evd>) (<evd_out>|-d <inline_evd>)
  <evm> --key=<key>
  [-c|--check-sort] [-i|--ignore-case] [-x|--text-input] [-y|--text-output]
  [-o|--output <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [-r|--reject <f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]
  [-v|--verbose]

evl aggreg
  ( --help | --usage | --version )

Options ¶

-c, --check-sort
check if the input is really sorted according to specified key

-D, --input-definition=<inline_evd>
either this option or the file <evd_in> must be presented. Example: ‘-D 'id int, user_id string'’

-d, --output-definition=<inline_evd>
either this option or the file <evd_out> must be presented. Example: ‘-d 'user_sum long'’

-i, --ignore-case
be case insensitive for key fields

-k, --key=<key>
group by this key, where <key> is comma separated list of fields with type (either DESC or ASC, default type is ASC). Example: ‘--key='id,user_id DESC'’

-o, --output=<f_out>
when output() function is used in the mapping, out structure is forwarded into <f_out>

--output<n>=<f_out>
when function ‘output(<n>)’ is used in mapping, where <n> is an integer from 4 to 16, out structure is forwarded into <f_out>

--outputs=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for output(N) functions in mapping. Example: for ‘--outputs=OUTFILE’, ‘${OUTFILE[120]}’ is the filename used for ‘output(120)’

-r, --reject=<f_out>
when reject() function is used in the mapping, input record is rejected into <f_out>

--reject<n>=<f_out>
when function reject(<n>) is used in mapping, where <n> is an integer from 4 to 16, input record is rejected into <f_out>

--rejects=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for reject(N) functions in mapping. Example: for ‘--rejects=REJECTS’, ‘${REJECTS[1000]}’ is the filename used for ‘reject(1000)’

-x, --text-input
suppose the input as text, not binary

-y, --text-output
write the output as text, not binary

Standard options: ¶

--help
print this help and exit

--usage
print short usage information and exit

-v, --verbose
print to stderr info/debug messages of the component

--version
print version and exit

Examples ¶

1. To print to stdout average of amount values:

   evl aggreg -D 'amount int' -d 'avg int' average.evm -k '' -xy <in.txt

   File ‘average.evm’ might look like this:

   VARIABLES:
   static int count;
   static long sum;
   
   INITIALIZE:
   count=0;
   sum=0;
   
   COMPUTE:
   sum += *in->amount;
   count++;
   
   FINALIZE:
   *out->avg = sum/count;

---

9.2 Join ¶

(since EVL 1.0)

Join <f_left> and <f_right> according to <key> and write to <f_out> or stdout. Inputs must be sorted by the <key>.

Synopsis ¶

Join
  <f_left> <f_right> <f_out>
  (<evd_left> | -L <inline_evd>) (<evd_right> | -R <inline_evd>)
  (<evd_out> | -d <inline_evd>) <evm> (-k <key> | -l <key> -r <key> )
  (-t|--type (left|right|inner|outer|cross))
  [-c|--check-sort] [-i|--ignore-case] [-y|--text-output]
  [ [-x|--text-input] | [-a|--left-text-input] [-b|--right-text-input] ]
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [--reject=<f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]

evl join
  <f_left> <f_right>
  (<evd_left> | -L <inline_evd>) (<evd_right> | -R <inline_evd>)
  (<evd_out> | -d <inline_evd>) <evm> (-k <key> | -l <key> -r <key> )
  (-t|--type (left|right|inner|outer|cross))
  [-c|--check-sort] [-i|--ignore-case] [-y|--text-output]
  [ [-x|--text-input] | [-a|--left-text-input] [-b|--right-text-input] ]
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [--reject=<f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]
  [-v|--verbose]

evl join
  ( --help | --usage | --version )

Options ¶

-L, --left-definition=<inline_evd>
either this option or the file <evd_left> must be presented

-R, --right-definition=<inline_evd>
either this option or the file <evd_right> must be presented

-c, --check-sort
check if the input is really sorted according to specified key

-d, --output-definition=<inline_evd>
either this option or the file <evd_out> must be presented. Example: ‘-d 'user_sum long'’

-i, --ignore-case
be case insensitive for key fields

-k, --key=<key>
join by this key, where <key> is comma separated list of fields with sort type (either DESC or ASC, default type is ASC). This is the shortcut for having the same lists of key fields for ‘--key-left’ and ‘--key-right’. Example: ‘--key='id,user_id DESC'’

-l, --key-left=<key>
comma separated list of left fields to join according to Example: ‘--key-left='id,name'’

-r, --key-right=<key>
comma separated list of right fields to join according to Example: ‘--key-right='user_id,surname'’

-t, --type=<type>
mandatory option specifying the join type, possible values for <type> are: ‘left’, ‘right’, ‘inner’, ‘outer’, ‘cross’

--unmatched-left=<f_out>
when ‘unmatched_left()’ function is used in the mapping, out structure is forwarded into <f_out>

--unmatched-right=<f_out>
when ‘unmatched_right()’ function is used in the mapping, out structure is forwarded into <f_out>

-o, --output=<f_out>
when ‘output()’ function is used in the mapping, out structure is forwarded into <f_out>

--output<n>=<f_out>
when function ‘output(<n>)’ is used in mapping, where <n> is an integer from 7 to 16, out structure is forwarded into <f_out>

--outputs=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for ‘output(N)’ functions in mapping. Example: for ‘--outputs=OUTFILE’, ‘${OUTFILE[120]}’ is the filename used for ‘output(120)’

--reject=<f_out>
when ‘reject()’ function is used in the mapping, input record is rejected into <f_out>

--reject<n>=<f_out>
when function ‘reject(<n>)’ is used in mapping, where <n> is an integer from 7 to 16, input record is rejected into <f_out>

--rejects=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for ‘reject(N)’ functions in mapping. Example: for ‘--rejects=REJECTS’, ‘${REJECTS[1000]}’ is the filename used for ‘reject(1000)’.

-a, --left-text-input
suppose the left input as text, not binary

-b, --right-text-input
suppose the right input as text, not binary

-x, --text-input
suppose the input as text, not binary

-y, --text-output
write the output as text, not binary

Standard options: ¶

--help
print this help and exit

--usage
print short usage information and exit

-v, --verbose
print to stderr info/debug messages of the component

--version
print version and exit

---

9.3 Map ¶

(since EVL 1.0)

Map input columns to output ones.

Map
is to be used in EVS job structure definition file. <f_in> and <f_out> are either input and output file or flow name.

evl map
is intended for standalone usage, i.e. to be invoked from command line and reading records from standard input and writing to standard output.

EVD, EVM and EVS are EVL definition files, for details see evl-evd(5), evl-evm(5) and evl-evs(5).

Synopsis ¶

Map
  <f_in> <f_out> (<evd_in>|-D <inline_evd>) (<evd_out>|-d <inline_evd>) <evm>
  [-x|--text-input] [-y|--text-output]
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [--reject <f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]

evl map
  (<evd_in>|-D <inline_evd>) (<evd_out>|-d <inline_evd>) <evm>
  [-x|--text-input] [-y|--text-output]
  [-o <f_out>] [--output<n>=<f_out>]... [--outputs=<varname>]
  [--reject=<f_out>] [--reject<n>=<f_out>]... [--rejects=<varname>]
  [-v|--verbose]

evl map
  ( --help | --usage | --version )

Options ¶

-D, --input-definition=<inline_evd>
either this option or the file <evd_in> must be presented. Example: ‘-D 'id int, user_id string'’

-d, --output-definition=<inline_evd>
either this option or the file <evd_out> must be presented. Example: ‘-d 'user_sum long'’

-o, --output=<f_out>
when ‘output()’ function is used in the mapping, out structure is forwarded into <f_out>

--output<n>=<f_out>
when function ‘output(<n>)’ is used in mapping, where <n> is an integer from 4 to 16, out structure is forwarded into <f_out>

--outputs=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for ‘output(N)’ functions in mapping. Example: for ‘--outputs=OUTFILE’, ‘${OUTFILE[120]}’ is the filename used for ‘output(120)’

--reject=<f_out>
when ‘reject()’ function is used in the mapping, input record is rejected into <f_out>

--reject<n>=<f_out>
when function ‘reject(<n>)’ is used in mapping, where <n> is an integer from 4 to 16, input record is rejected into <f_out>

--rejects=<varname>
specifies an array ‘${<varname>[@]}’ which contains filenames to be used for ‘reject(N)’ functions in mapping. Example: for ‘--rejects=REJECTS’, ‘${REJECTS[1000]}’ is the filename used for ‘reject(1000)’

-x, --text-input
suppose the input as text, not binary

-y, --text-output
write the output as text, not binary

Standard options: ¶

--help
print this help and exit

--usage
print short usage information and exit

-v, --verbose
print to stderr info/debug messages of the component

--version
print version and exit

---