Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a build step to deploy to EKS #11231

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
+23 −18
Open

Add a build step to deploy to EKS #11231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5a9bd98
Add a build step to deploy to EKS
voidlily Sep 12, 2024
cd51557
Get rid of the misnamed `deploy_production` step entirely
voidlily Sep 18, 2024
d1a98b4
Also check for pgbouncer by checking if 'pgbouncer' is in the hostname
voidlily Sep 18, 2024
936429f
make prepared statements a config value rather than a hack of checkin…
voidlily Sep 18, 2024
92ac23d
config updates
mitchellhenke Sep 19, 2024
a39eeb9
fix logic, || should be &&
voidlily Sep 19, 2024
118043d
only use the new database_prepared_statement and related configs
voidlily Sep 19, 2024
79519a1
add _enabled suffix
mitchellhenke Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 15 additions & 16 deletions .gitlab-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ stages:
- after_test
- review
- scan
- deploy_production
- deploy_eks

workflow:
rules:
Expand Down Expand Up @@ -604,21 +604,6 @@ stop-review-app:
- if: $CI_PIPELINE_SOURCE != "merge_request_event"
when: never

deploy_production:
stage: deploy_production
allow_failure: true
needs:
- job: build-review-image
resource_group: $CI_ENVIRONMENT_SLUG.reviewapps.identitysandbox.gov
extends: .deploy
environment:
name: production
deployment_tier: production
url: https://$CI_ENVIRONMENT_SLUG.reviewapps.identitysandbox.gov
rules:
- if: $CI_COMMIT_BRANCH == "main" && $CI_PIPELINE_SOURCE == "push"


include:
- template: Jobs/SAST.gitlab-ci.yml
- template: Jobs/Dependency-Scanning.gitlab-ci.yml
Expand Down Expand Up @@ -867,3 +852,17 @@ audit_packages_scheduled:
fi
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"

# EKS deployment
deploy_eks:
trigger:
project: lg-public/identity-eks-control
# TODO change branch to main before merge
branch: lrappaport/deploy-eks
stage: deploy_eks
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific job this stage depends on? Can we use needs to be more precise?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the use of needs, the dependency is "all jobs in previous stages"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at https://gitlab.login.gov/lg/identity-idp/-/pipelines/120712 you can see it's currently the rightmost stage

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, sorry, I understand that aspect. This job is dependent on deploy_production?

I didn't quite understand the interaction with the deploy to "production" there with the description of deploying to different environments in the PR description.

Copy link
Member

@timothy-spencer timothy-spencer Sep 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we should seriously consider renaming the deploy_production job to deploy_reviewapp or something. It's not production, and definitely can be confusing. Probably also need to remove it's environment.name: production stuff too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's kind of why I lean towards using needs to explicitly define the dependencies. I think it's worth renaming deploy_production as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i guess, my main requirements here are "everything" and if later on we add another step to the test phase i want that to automatically get picked up as a requirement? hence the stages thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, doesn't the main reviewapp go away now? it would be the same as a continuously deployed dev env that this pipeline also adds?

https://idp.eksdev.identitysandbox.gov/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there's no objection i'm just gonna delete that step entirely

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd defer to @timothy-spencer on that aspect.

variables:
APP: idp
IMAGE_TAG: $CI_COMMIT_SHA
# TODO uncomment before merge, this is for testing on branch
# rules:
# - if: $CI_COMMIT_BRANCH == "main"
2 changes: 2 additions & 0 deletions config/application.yml.default
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,10 +62,12 @@ component_previews_enabled: false
compromised_password_randomizer_threshold: 900
compromised_password_randomizer_value: 1000
country_phone_number_overrides: '{}'
database_advisory_locks_enabled: false
database_host: ''
database_name: ''
database_password: ''
database_pool_idp: 5
database_prepared_statements_enabled: false
database_read_replica_host: ''
database_readonly_password: ''
database_readonly_username: ''
Expand Down
2 changes: 2 additions & 0 deletions config/application.yml.default.docker
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ production:
available_locales: 'en,es,fr,zh'
asset_host: ['env', 'ASSET_HOST']
component_previews_enabled: true
database_advisory_locks_enabled: false
database_host: ['env', 'POSTGRES_HOST']
database_name: ['env', 'POSTGRES_NAME']
database_prepared_statements_enabled: false
database_password: ['env', 'POSTGRES_PASSWORD']
database_sslmode: ['env', 'POSTGRES_SSLMODE']
database_username: ['env', 'POSTGRES_USERNAME']
Expand Down
4 changes: 2 additions & 2 deletions config/database.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ production:
host: <%= IdentityConfig.store.database_socket.present? ? IdentityConfig.store.database_socket : IdentityConfig.store.database_host %>
password: <%= IdentityConfig.store.database_password %>
pool: <%= primary_pool %>
advisory_locks: <%= !IdentityConfig.store.database_socket.present? %>
prepared_statements: <%= !IdentityConfig.store.database_socket.present? %>
advisory_locks: <%= IdentityConfig.store.database_advisory_locks_enabled %>
prepared_statements: <%= IdentityConfig.store.database_prepared_statements_enabled %>
sslmode: <%= IdentityConfig.store.database_sslmode %>
sslrootcert: '/usr/local/share/aws/rds-combined-ca-bundle.pem'
migrations_paths: db/primary_migrate
Expand Down
2 changes: 2 additions & 0 deletions lib/identity_config.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,12 @@ def self.store
config.add(:country_phone_number_overrides, type: :json)
config.add(:dashboard_api_token, type: :string)
config.add(:dashboard_url, type: :string)
config.add(:database_advisory_locks_enabled, type: :boolean)
config.add(:database_host, type: :string)
config.add(:database_name, type: :string)
config.add(:database_password, type: :string)
config.add(:database_pool_idp, type: :integer)
config.add(:database_prepared_statements_enabled, type: :boolean)
config.add(:database_read_replica_host, type: :string)
config.add(:database_readonly_password, type: :string)
config.add(:database_readonly_username, type: :string)
Expand Down