Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add SFN driven rosbag-image-pipeline module #403

Merged

Conversation

malachi-constant
Copy link
Contributor

@malachi-constant malachi-constant commented Feb 2, 2024

  • ASL -> CDK Conversion
  • CDK Synth passing
  • CDK Deploy passing
  • Functional Testing
  • seed-farmer integration testing
  • Security & Unit tests

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@malachi-constant malachi-constant self-assigned this Feb 2, 2024
@malachi-constant malachi-constant marked this pull request as draft February 2, 2024 18:54
@dgraeber dgraeber self-requested a review February 9, 2024 17:54
Copy link
Contributor

@dgraeber dgraeber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we should make this a generic module...keep it in ADDF, but it may be better to make all new modules generic so it can be leveraged in other projects?

Lets debate

modules/analysis/rosbag-image-pipeline-sfn/deployspec.yaml Outdated Show resolved Hide resolved
@malachi-constant
Copy link
Contributor Author

Wondering if we should make this a generic module...keep it in ADDF, but it may be better to make all new modules generic so it can be leveraged in other projects?

Lets debate

Depends how generic you mean..

A good 90% of this code is setting up what was in the DAG. So if you mean bringing your own logic for EMR, batch, sagemaker, yes.

For just SFN usage, doesn't quite make sense.

@malachi-constant malachi-constant marked this pull request as ready for review February 9, 2024 21:22
@malachi-constant malachi-constant marked this pull request as draft February 9, 2024 21:26
@malachi-constant malachi-constant added bug Something isn't working feature and removed bug Something isn't working labels Feb 9, 2024
dgraeber

This comment was marked as resolved.

@dgraeber

This comment was marked as resolved.

@malachi-constant malachi-constant marked this pull request as ready for review February 13, 2024 20:44
@malachi-constant malachi-constant dismissed dgraeber’s stale review February 20, 2024 00:25

Changes Addressed, requesting re-review

Copy link
Contributor

@dgraeber dgraeber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to address the accountID being committed....the other comments are nice to have

modules/analysis/rosbag-image-pipeline-sfn/stack.py Outdated Show resolved Hide resolved
manifests/sfn-example/deployment.yaml Outdated Show resolved Hide resolved
modules/analysis/rosbag-image-pipeline-sfn/deployspec.yaml Outdated Show resolved Hide resolved
@malachi-constant malachi-constant merged commit 6a0f654 into awslabs:main Feb 22, 2024
69 checks passed
@malachi-constant malachi-constant deleted the feat/refactor-pipeline-to-use-sfn branch February 22, 2024 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants