Parameterize import artifacts dirs
Currently, in the post_init for anything inheriting from AbstractFileArtifact
and ContainerArtifact
, we're forcing static destination paths as such.
Dest path is set here initially, where it can be set either as a parameter passed to the class, or as an environment variable. I think this is fine since we can still update the path dynamically.
@dataclass
class AbstractArtifact(ABC):
def __post_init__(self):
self.dest_path = self.dest_path or pathlib.Path(
f"{os.environ.get('ARTIFACT_DIR')}"
)
In the following two classes, the final destination dir is set statically to either <base_dest_dir>/external-resources
or <base_dest_dir>/images
class AbstractFileArtifact(AbstractArtifact):
def __post_init__(self):
super().__post_init__()
# self.dest_path passed to pathlib is set in AbstractArtifact
self.dest_path = pathlib.Path(self.dest_path, "external-resources")
and
class ContainerArtifact(AbstractArtifact):
def __post_init__(self):
super().__post_init__()
# self.dest_path passed to pathlib is set in AbstractArtifact
self.dest_path = pathlib.Path(self.dest_path, "images")
We should allow these fields to be set when initializing the class, rather than enforcing all artifacts need to go into either external-resources
and images
. We might also want to consider updating this code to make it less confusing how the destination directory is set through inheritance.