Sequencer

class bonobo_trans.sequencer.Sequencer(*args, **kwargs)

The Sequencer transformation is a number generator.

Configuration options

Optional:

  • name (str, length max. 30)
  • sequence_key (str) Default: SEQ
  • initial (int) Default: 1
  • increment (int) Default: 1
  • max (int)
  • cycle (bool) Default: False
  • generator (bool) Default: False
  • generate (int)
  • source_value_col (str)
  • source_value_tbl (str)
  • persist_type (int) Default:
  • persist_table (str)
  • persist_file (str)

Option descriptions:

name
Name of the transformation. Required when using persistence.
sequence_key
Name of the sequence key in the outgoing grow. Default is ‘SEQ’.
initial
Starting value. Will start at 1 if not specified.
increment
Value to add in every increment. Will increment by 1 if not specified.
max
Maximum allowed value. When reached the sequencer will stop generating new numbers. If the ‘cycle’ option is True, the sequencer will restart at the initial value.
cycle
When set to True, the sequencer will restart at the initial value after reaching the max value.
source_value_tbl, source_value_col
Use to retrieve an initial value from an existing table. See notes below.

Note

Row generation

generator, generate
Use to generate rows instead of appending. See notes below.
persist_type, persist_file, persist_table
Persist sequence values. See notes below.
source_value_tbl, source_value_col
It’s possible to start with an initial value based on an existing value in a database table. Provide the table and column name using the source_value_tbl and source_value_col-options.
generator, generate

Instead of appending a row with a sequence number it is possible to generate a set of rows instead. To do so, set the ‘generator’ option to True and the ‘generate’ option to the number of rows you want to generate.

The generator mode is essentialy an “extract” transformation, and as such, no rows can be passed onto it.

By default the generator mode is not enabled.

Note

Persistence

Persistence enables the sequencer to continue the sequence after restarting. The current value will need to be stored in a database or in a file.

By default persistence is not enabled.

There is no mechanism to remove unused files, tables or table entries. You will need to clean-up these using ?? How to add utility functions to this class ??

persist_type
persist_type Description
SEQ_PERSIST_DISABLED No persistence.
SEQ_PERSIST_DB Persist to a DB table.
SEQ_PERSIST_FILE Persist to a flatfile.
persist_file
When using SEQ_PERSIST_FILE, the persist_file option will need to hold the fully qualifed path and file name to which to save the sequence value.
persist_table, persist_allow_creation
When using SEQ_PERSIST_DB, the persist_table option will need to hold the table name to which to write the sequence value. If the table does not exist and ‘persist_allow_creation’ is True, the table will be created automatically. When creating the table in advance, you must include the following fields: - sequence_name string(30) - sequence_nr numeric
Args:
  • d_row_in (dict)
Returns:
  • d_row_out (dict)

d_row_out contains all the keys of the incoming dictionary plus the sequencer key (set using the ‘sequence_key’-option). If there already is a key with that name it will be overwritten.

Parameters:
  • engine (str) –
  • name (str) –
  • sequence_key (str) –
  • source_value_tbl (str) –
  • source_value_col (str) –
  • initial (int) –
  • increment (int) –
  • max (int) –
  • cycle (bool) –
  • persist_type (int) –
  • persist_table (str) –
  • persist_file (str) –
  • persist_allow_creation (bool) –
  • generator (bool) –
  • generate (int) –