Skip to content

System Overview

Contents

djtools as a CLI

djtools is a set of packages that provide functional interfaces to streamline tasks.

Although there are parts of djtools, like the collection package, that are designed to operate interactively, djtools is primarily intended to be used as a CLI.

In a nutshell, the main function of djtools builds a configuration, iterates through the top-level operations exported by each package, and executes those operations as determined by the configuration.

When building the configuration, djtools first looks for a config.yaml in the configs folder from which to load the configuration objects. If no config file is found, default values defined in the Pydantic models are used. If CLI arguments are provided, they override the corresponding configuration options:

Source code in djtools/configs/helpers.py 
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
@make_path
def build_config(
    config_file: Path = Path(__file__).parent / "config.yaml",
) -> BaseConfig:
    """This function loads configurations for the library.

    Configurations are loaded from config.yaml. If command-line arguments are
    provided, these override the configuration options set in config.yaml.

    Args:
        config_file: Optional path to a config.yaml.

    Raises:
        RuntimeError: config.yaml must be a valid YAML.

    Returns:
        Global configuration object.
    """
    # Create a default config if one doesn't already exist.
    if not config_file.exists():
        with open(config_file, mode="w", encoding="utf-8") as _file:
            yaml.dump(BaseConfig().model_dump(), _file)

    # Load the config.
    try:
        with open(config_file, mode="r", encoding="utf-8") as _file:
            config_data = yaml.load(_file, Loader=yaml.FullLoader) or {}
        config = BaseConfig(**config_data)
    except Exception as exc:
        msg = f"Error reading config file {config_file}: {exc}"
        logger.critical(msg)
        raise ConfigLoadFailure(msg) from exc

    entry_frame_filename = inspect.stack()[-1][1]
    valid_filenames = (
        str(Path("bin") / "djtools"),  # Unix djtools.
        str(Path("bin") / "pytest"),  # Unix pytest.
        str(Path("lib") / "runpy.py"),  # Windows Python<=3.10.
        "<frozen runpy>",  # Windows Python>=3.11.
    )

    # Only get CLI arguments if calling djtools as a CLI.
    if entry_frame_filename.endswith(valid_filenames):
        cli_args = {
            k: v for k, v in _arg_parse().items() if v or isinstance(v, list)
        }
        logger.info(f"Args: {cli_args}")
        config = _update_config_with_cli_args(config, cli_args)

    return config

At the end of config building, a BaseConfig object containing the union of the package configs' options is returned. The top-level operations exported by the different djtools packages all take a BaseConfig object as an argument.

djtools as a Library

The djtools library is organized into 5 packages:

  • configs
  • collection
  • spotify
  • sync
  • utils

Each of the packages collection, spotify, sync, and utils export the module functions that implement the top-level features of djtools.

Additionally, the configs package exports build_config which generates the BaseConfig object that must be passed into all of the other exported functions.

The exception are the *Collection, *Playlist, and *Track classes exported from the collection package. These are exported as a convenience for interactive collection manipulation in a Python interpreter.

Note that, while build_config accepts an optional path to a config.yaml, features that interact with the other YAML configs (collection_playlists.yaml and spotify_playlists.yaml) still reference those config paths relative to the source.

CI

There are several GitHub Actions bound to the lifecycle of djtools.

The pytest-coverage and pylint workflows are triggered on pull request events. Passing both of these workflows is a check for merges into release branches or main. A pass rate of 100% is required.

On pushes to release/** branches, changes to pyproject.toml or .py files trigger the release-dev workflow which performs a pre-release version bump and wheel release. Changes to .md files trigger the deploy-dev-docs workflow which copies files to a shadow repository and deploys docs on that repo's GitHub Pages.

On pushes to main, the release-prod and deploy-prod-docs workflows are triggered which perform essentially the same steps as the equivalent dev workflows except the version is finalized instead of bumped and the docs are deployed on the main repository's GitHub Pages.