Use poetry to package our modules
To remove the requirement for us to add sys.path.append(...
to all of our files to allow relative imports from other directories, we need to package our modules directory with poetry and update our pipeline yaml to pip install the modules directory as a dependency.
The existing branch should have the main setup to get started, but we'll still need to do the following.
-
Remove sys.path.appends
for imports -
Update all imports to use ironbank.pipeline
relative import path -
Update all module and stage python file to allow execution (i.e. chmod +x
) -
Update all pipeline .yaml files to install the ironbank package where applicable -
python3 -m pip install ironbank-pipeline
will install theironbank
package
-
-
Consider updating READMEs to explain updated dev and prod workflows
Short term plan: Prod:
- pip install ironbank-pipeline dir
Dev:
- poetry install
- poetry shell or poetry run (not working on my system atm)
- add all deps as major version deps
Longterm: Prod:
- Build into a package or a container
Edited by Kenneth Maguire