Components-Common

7 Components Common ¶

ETL jobs are defined in evs files and consist of components connected by flows.

In this chapter, all components are described.

There are two kinds of component commands:

• ‘Component’, i.e. component name begins with capital letter – these are intended to be used in evs file (EVL job structure files),
• ‘evl component’, i.e. command ‘evl’ with the first argument to be a name of the component – these invocations are to be used as standalone commands, which can be run from commandline.

7.1 Common Options ¶

Mostly this invocation schema is applied on all components.

<Component>  F_IN... F_OUT... EVD_IN... EVD_OUT... EVM
        [ -v|--validate ]
        [ -x|--text-input ]
        [ -y|--text-output ]

where <Component> is the name of the component, like Read or Join.

F_IN... F_OUT...
are names of the input flows or files and of the output flows or files.

• Flow names must be unique in each evs file. Convention is to have these names in CAPITAL LETTERS.
• Files – if F_IN or F_OUT contains ‘/’ (foreslash), then it is treated as a file, not a flow. Remember that one can use special files, like /dev/stderr or /dev/null or in general /dev/fd/N.

EVD_IN... EVD_OUT...
specifies the evd with the information about data structure, i.e. field names, data types, nullability, separator, etc. It can be:

• EVD file – a file with the data structure definition;

• Inline EVD – evd can be specified inline by options this way: -D | --input-definition
  for input evd (when both – input and output – evd have to be specified);

  -d | --output-definition
  for output evd (when both – input and output – evd have to be specified);

  -d | --data-definition
  for input evd (whenonly one evd is needed by <Component>).

EVM
specifies the evm file which contains the mapping of the component.

All options are interpreted as in shell, so must be specified in one line or separated by ‘\’ at the end of the line.

---