Delta Tables Test Stubs

Delta table test classes.

Merges operations will be just logged with no change to the database.

spark_tests.delta.FAKE_DELTA_MERGE = <spark_tests.delta.FakeDeltaMerge object>: FakeDeltaMerge singleton instance.

class spark_tests.delta.FakeDeltaMerge

Log of a merge operation.

Logs merge parameters to check later in a test case, but does not execute the merge.

target: Target DeltaTable

source: DataFrame Source DataFrame

condition: Condition to match source rows with the target rows

matched_action: {“UPDATE”, “DELETE”} Optional action to be done on matched rows

matched_condition: Optional condition to perform not_matched_action

matched_update: Defines the rules of setting the values of columns that need to be updated.

not_matched_action: {“INSERT”} Optional action to be done on not matched rows

not_matched_condition: Optional condition to perform not_matched_action

not_matched_insert: Defines the rules of setting the values of columns that need to be inserted.

is_executed: Flag of execution.

clear() → None: Clear self.

class spark_tests.delta.FakeDeltaMergeBuilder(target: FakeDeltaTable, source: FakeDataFrame, condition: str)

Builds a FakeDeltaMerge.

execute() → None

Execute the merge operation based on the built matched and not matched actions.

See DeltaMergeBuilder for complete usage details.

New in version 0.4.

whenMatchedDelete(condition: Optional[str] = None) → FakeDeltaMergeBuilder

Delete a matched row from the table only if the given condition (if specified) is true for the matched row.

See DeltaMergeBuilder for complete usage details.

Parameters: condition (str or pyspark.sql.Column) – Optional condition of the delete
Returns: this builder

New in version 0.4.

whenMatchedUpdate(condition: Optional[str] = None, set: Optional[Dict[str, str]] = None) → FakeDeltaMergeBuilder

Update a matched table row based on the rules defined by set. If a condition is specified, then it must evaluate to true for the row to be updated.

See DeltaMergeBuilder for complete usage details.

Parameters

condition (str or pyspark.sql.Column) – Optional condition of the update
set (dict with str as keys and str or pyspark.sql.Column as values) – Defines the rules of setting the values of columns that need to be updated. Note: This param is required. Default value None is present to allow positional args in same order across languages.

Returns

this builder

New in version 0.4.

whenNotMatchedInsert(condition: Optional[str] = None, values: Optional[Dict[str, str]] = None) → FakeDeltaMergeBuilder

Insert a new row to the target table based on the rules defined by values. If a condition is specified, then it must evaluate to true for the new row to be inserted.

See DeltaMergeBuilder for complete usage details.

Parameters

condition (str or pyspark.sql.Column) – Optional condition of the insert
values (dict with str as keys and str or pyspark.sql.Column as values) – Defines the rules of setting the values of columns that need to be updated. Note: This param is required. Default value None is present to allow positional args in same order across languages.

Returns

this builder

New in version 0.4.

class spark_tests.delta.FakeDeltaTable(spark: FakeSparkSession)

Logs merge operations with no change in the data.

spark: FakeSparkSession

name: Optional name of the table

path: Optional path of the table

alias_name: alias name of the table

source: DataFrame to merge

alias(alias_name: str) → FakeDeltaTable: Sets alias for self.

classmethod forName(spark: FakeSparkSession, name: str) → FakeDeltaTable: Creates a FakeDeltaTable for given name.

classmethod forPath(spark: FakeSparkSession, path: str) → FakeDeltaTable: Creates a FakeDeltaTable for given path.

merge(source: FakeDataFrame, condition: str) → FakeDeltaMergeBuilder: Start building FakeDeltaMerge from source DataFrame on the given merge condition.