legenddataflowscripts.workflow package¶

Submodules¶

legenddataflowscripts.workflow.execenv module¶

legenddataflowscripts.workflow.execenv._execenv2str(cmd_expr, cmd_env)¶

Return type:: str

legenddataflowscripts.workflow.execenv.apptainer_env_vars(cmdenv)¶

Return type:: list[str]

legenddataflowscripts.workflow.execenv.cmdexec(args)¶

Load the data production environment and execute a given command.

legenddataflowscripts.workflow.execenv.dataflow()¶

dataflow’s CLI for installing and loading the software in the data production environment.

$ dataflow --help
$ dataflow install --help  # help section for a specific sub-command

legenddataflowscripts.workflow.execenv.execenv_prefix(config, as_string=True)¶

Returns the software environment command prefix.

For example: apptainer run image.sif

Note

If as_string is True, a space is appended to the returned string.

Return type:: str | tuple[list, dict]

legenddataflowscripts.workflow.execenv.execenv_pyexe(config, exename, as_string=True)¶

Returns the path to an executable installed in the virtualenv.

For example: apptainer run image.sif path/to/venv/bin/{exename}

Note

If as_string is True, a space is appended to the returned string.

Return type:: str | tuple[list, dict]

legenddataflowscripts.workflow.execenv.install(args)¶

Installs user software in the data production environment.

The software packages should be specified in the config_file with the format:

pkg_versions:
  - python_package_spec
  - ...

$ dataflow install config.yaml
$ dataflow install --editable config.yaml  # install legend-dataflow in editable mode
$ dataflow install --remove config.yaml  # remove install directory

legenddataflowscripts.workflow.execenv.oci_engine_env_vars(cmdenv)¶

Return type:: list[str]

legenddataflowscripts.workflow.filedb module¶

legenddataflowscripts.workflow.filedb.build_filedb()¶

legenddataflowscripts.workflow.pre_compile_catalog module¶

legenddataflowscripts.workflow.pre_compile_catalog.pre_compile_catalog(validity_path)¶

legenddataflowscripts.workflow.utils module¶

This module contains all the utility needed for the data production. for substituting the pathvar in the config, also the conversion from timestamp to unix time

legenddataflowscripts.workflow.utils.as_ro(config, path)¶

legenddataflowscripts.workflow.utils.set_last_rule_name(workflow, new_name)¶: Sets the name of the most recently created rule to be new_name. Useful when creating rules dynamically (i.e. unnamed).

Warning

This could mess up the workflow. Use at your own risk.

legenddataflowscripts.workflow.utils.subst_vars(props, var_values=None, use_env=False, ignore_missing=False)¶

legenddataflowscripts.workflow.utils.subst_vars_impl(x, var_values, ignore_missing=False)¶

legenddataflowscripts.workflow.utils.subst_vars_in_snakemake_config(workflow, config)¶