YAML Schema Reference

Complete YAML configuration specification for GoMask CLI

YAML Schema Reference

Complete specification for GoMask CLI YAML configuration files.

Routine Types

GoMask supports two routine types:

Type	Purpose
`synthetic`	Generate new synthetic test data
`masking`	Mask/anonymize existing data

Top-Level Structure

Both routine types share a common top-level structure:

routine:                    # Required: Routine configuration
  name: "Display Name"
  type: synthetic           # or "masking"
  connector_id: 123
  description: "Optional description"

settings:                   # Optional: Execution settings
  batch_size: 1000

tables:                     # Required: Table configurations
  - table_name: users
    # ... table config

SyntheticRoutine

Full schema for synthetic data generation:

routine:
  name: "Customer Data Generator"
  type: synthetic
  connector_id: 1
  description: "Generates synthetic customer data"

settings:
  generation_mode: hierarchical
  batch_size: 1000
  parallel_workers: 4
  enforce_referential_integrity: true
  global_record_count: 1000
  localization_language: en
  localization_region: US
  runtime_parameter_definitions:
    - key: "param_record_count"
      name: "record_count"
      type: integer
      defaultValue: 1000
      description: "Number of records to generate"
      required: false

tables:
  - table_name: users
    schema_name: public
    hierarchy_level: 0
    target_record_count: 1000
    order_index: 0
    columns:
      - column_name: id
        generation_function: sequential_integer
        is_primary_key: true
        generation_parameters:
          - name: start
            value: 1
      - column_name: email
        generation_function: generate_email
        is_unique: true
        generation_parameters:
          - name: domain
            value: "example.com"
      - column_name: first_name
        generation_function: generate_first_name
      - column_name: created_at
        generation_function: generate_datetime
        generation_parameters:
          - name: start
            value: "2023-01-01"
          - name: end
            value: "2024-12-31"

  - table_name: orders
    schema_name: public
    hierarchy_level: 1
    parent_table_name: users
    target_record_count: 5000
    record_distribution:
      distribution_type: uniform
      min_records_per_parent: 1
      max_records_per_parent: 10
    columns:
      - column_name: id
        generation_function: sequential_integer
        is_primary_key: true
      - column_name: user_id
        is_foreign_key: true
        referenced_table: users
        referenced_column: id
      - column_name: total
        generation_function: generate_decimal
        generation_parameters:
          - name: min
            value: 10.00
          - name: max
            value: 500.00
          - name: precision
            value: 2

MaskingRoutine

Full schema for data masking:

routine:
  name: "PII Masking"
  type: masking
  connector_id: 1
  description: "Anonymize sensitive customer data"

settings:
  batch_size: 1000
  parallel_workers: 4
  audit_reporting_enabled: true
  localization_language: en
  localization_region: US

tables:
  - table_name: customers
    schema_name: public
    columns:
      - column_name: email
        masking_function: mask_email
        sensitivity_level: high
        data_category: pii
        masking_config:
          - name: preserve_domain
            value: false
      - column_name: ssn
        masking_function: mask_ssn
        sensitivity_level: high
        masking_config:
          - name: preserve_last
            value: 4
      - column_name: phone
        masking_function: mask_phone
        masking_config:
          - name: preserve_format
            value: true
      - column_name: first_name
        masking_function: generate_first_name
      - column_name: notes
        masking_function: nullify

Routine Section

The routine section defines the routine metadata:

Field	Type	Required	Description
`name`	string	Yes	Display name of the routine
`type`	string	Yes	`synthetic` or `masking`
`connector_id`	integer	Yes	ID of the database connector
`description`	string	No	Optional description

Settings Section

Synthetic Settings

Field	Type	Default	Description
`generation_mode`	string	`hierarchical`	Generation mode
`batch_size`	integer	`1000`	Records per batch
`parallel_workers`	integer	`4`	Number of parallel workers
`enforce_referential_integrity`	boolean	`true`	Enforce FK constraints
`global_record_count`	integer	`1000`	Default record count
`localization_language`	string	`en`	Language code
`localization_region`	string	`US`	Region code
`runtime_parameter_definitions`	array	`[]`	Runtime parameters

Masking Settings

Field	Type	Default	Description
`batch_size`	integer	`1000`	Records per batch
`parallel_workers`	integer	`4`	Number of parallel workers
`audit_reporting_enabled`	boolean	`true`	Enable audit logs
`localization_language`	string	`en`	Language code
`localization_region`	string	`US`	Region code

Tables Section

Each table in the tables array has these properties:

Common Table Properties

Field	Type	Required	Description
`table_name`	string	Yes	Name of the database table
`schema_name`	string	No	Schema name (default: `public`)

Synthetic Table Properties

Field	Type	Default	Description
`hierarchy_level`	integer	`0`	Level in generation hierarchy
`parent_table_name`	string	-	Parent table for FK relationships
`target_record_count`	integer	-	Number of records to generate
`order_index`	integer	`0`	Order within hierarchy level
`table_type`	string	`entity`	Type: `entity` or `junction`
`record_distribution`	object	-	Distribution settings for child records

Record Distribution

Control how child records are distributed across parent records:

record_distribution:
  distribution_type: uniform    # uniform, normal, weighted
  min_records_per_parent: 1
  max_records_per_parent: 10
  average_records_per_parent: 5  # for normal distribution
  distribution_weights:         # for weighted distribution
    weights: [0.2, 0.5, 0.3]
    values: [1, 3, 5]
  custom_distribution_function: "lambda parent_id: random.randint(1, 10)"

Field	Type	Description
`distribution_type`	string	`uniform`, `normal`, or `weighted`
`min_records_per_parent`	integer	Minimum child records per parent
`max_records_per_parent`	integer	Maximum child records per parent
`average_records_per_parent`	number	Average for normal distribution
`distribution_weights`	object	Weights and values for weighted distribution
`custom_distribution_function`	string	Custom Python lambda for distribution

Columns Section

Synthetic Column Properties

Field	Type	Required	Description
`column_name`	string	Yes	Name of the column
`generation_function`	string	No	Function to generate data
`generation_parameters`	array	No	Function parameters
`data_type`	string	No	Database data type (alias: `original_data_type`)
`is_primary_key`	boolean	`false`	Is this the primary key
`is_foreign_key`	boolean	`false`	Is this a foreign key
`referenced_table`	string	No	FK referenced table (alias: `reference_table`)
`referenced_column`	string	No	FK referenced column (alias: `reference_column`)
`is_nullable`	boolean	`true`	Allow NULL values (alias: `nullable`)
`is_unique`	boolean	`false`	Unique constraint (alias: `unique_constraint`)
`is_indexed`	boolean	`false`	Has database index
`max_length`	integer	No	Maximum length for varchar/char
`precision`	integer	No	Precision for numeric columns
`scale`	integer	No	Scale for numeric columns
`default_value`	string	No	Default value for column
`enum_values`	array	No	Allowed enum values
`column_comment`	string	No	Column comment/description
`order_index`	integer	`0`	Column order in generation
`is_excluded`	boolean	`false`	Exclude from generation
`enabled`	boolean	`true`	Include in generation (inverse of `is_excluded`)

Masking Column Properties

Field	Type	Required	Description
`column_name`	string	Yes	Name of the column
`masking_function`	string	No	Function to mask data
`masking_config`	array	No	Function parameters
`sensitivity_level`	string	`low`	Data sensitivity: `low`, `medium`, `high`
`data_category`	string	`generic`	Category: `pii`, `financial`, etc.
`data_type`	string	No	Database data type (alias: `original_data_type`)
`is_nullable`	boolean	`true`	Allow NULL values (alias: `nullable`)
`is_primary_key`	boolean	`false`	Is this the primary key
`is_foreign_key`	boolean	`false`	Is this a foreign key
`is_unique`	boolean	`false`	Unique constraint (alias: `unique_constraint`)
`is_indexed`	boolean	`false`	Has database index
`max_length`	integer	No	Maximum length for varchar/char
`precision`	integer	No	Precision for numeric columns
`scale`	integer	No	Scale for numeric columns
`default_value`	string	No	Default value for column
`enum_values`	array	No	Allowed enum values
`column_comment`	string	No	Column comment/description
`referenced_table`	string	No	FK referenced table (alias: `reference_table`)
`referenced_column`	string	No	FK referenced column (alias: `reference_column`)
`is_excluded`	boolean	`false`	Exclude from masking
`enabled`	boolean	`true`	Include in masking (inverse of `is_excluded`)

Masking Config Parameters

The masking_config array supports the same parameter format as generation parameters:

masking_config:
  # Static value
  - name: preserve_domain
    value: false

  # Column reference (use value from another column)
  - name: source_value
    valueType: column
    columnReference: original_email

  # Table reference
  - name: lookup_value
    valueType: reference
    tableReference: lookup_table
    columnReference: masked_value
    mode: random
    filter: "category = 'email'"

Field	Type	Description
`name`	string	Parameter name (required)
`value`	any	Static parameter value
`valueType`	string	`static`, `reference`, `expression`, or `column`
`columnReference`	string	Column to reference
`tableReference`	string	Table to reference
`mode`	string	Reference mode: `random`, `sequential`
`filter`	string	Filter condition for reference lookup

Generation Parameters

Parameters are specified as an array of objects:

generation_parameters:
  - name: min
    value: 1
  - name: max
    value: 100
  - name: domain
    value: "example.com"

Parameter Value Types

generation_parameters:
  # Static value
  - name: prefix
    value: "CUST-"
    valueType: static

  # Column reference
  - name: source
    valueType: reference
    columnReference: first_name
    tableReference: users

  # Expression
  - name: calculated
    valueType: expression
    value: "row.quantity * row.price"

Runtime Parameters

Define parameters that can be overridden at execution time:

settings:
  runtime_parameter_definitions:
    - key: "param_record_count"
      name: "record_count"
      type: integer
      defaultValue: 1000
      description: "Number of records to generate"
      required: false
    - key: "param_start_date"
      name: "start_date"
      type: string
      defaultValue: "2024-01-01"
      description: "Start date for date ranges"

Use in column configurations:

tables:
  - table_name: users
    target_record_count: ${record_count}

Override at runtime:

gomask run routine.yaml --param record_count=5000

Validation

Validate your configuration:

# Basic validation
gomask validate routine.yaml

# With detailed errors
gomask validate routine.yaml --detailed

# Show parsed configuration
gomask validate routine.yaml --show-config

YAML Schema Reference