Simplify the package

username-removed-652089 requested to merge refactor-0.1 into master

Simplifies the package:

  • 🔥 the vanilla package, runners, and storages; focus on datasets and transforms
  • 🔥 configs and secrets; configs are defined in the classes for that dataset, secrets can be pulled in from there from an EJSON or whatever the user is using, not part of this package anymore
  • Add better test coverage for datasets and transforms
  • No more separate packages for dojo, dojo-beam, dojo-google
  • Use the stored package for interaction with the storage backend (local or GS)

TODO pre-merge:

  • Add jsonschema validation on input and output
  • Add error handler
  • Update downstream warehouse to use this version of the package
Edited by username-removed-652089

Merge request reports