Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support pod template for Spark 3.x applications #2141

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
+525 −27
Open

Support pod template for Spark 3.x applications #2141

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c7208f3
Update API definition to support pod template
ChenYi015 Aug 22, 2024
efd9eeb
Support pod template
ChenYi015 Aug 22, 2024
dc671cb
Add example for using pod template
ChenYi015 Aug 22, 2024
a7c5002
Mark pod template field as schemaless
ChenYi015 Aug 27, 2024
e78f6b6
Merge branch 'master' into feature/pod-template
yuchaoran2011 Sep 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ version: ## Print version information.

.PHONY: manifests
manifests: controller-gen ## Generate CustomResourceDefinition, RBAC and WebhookConfiguration manifests.
$(CONTROLLER_GEN) crd rbac:roleName=spark-operator-controller webhook paths="./..." output:crd:artifacts:config=config/crd/bases
$(CONTROLLER_GEN) crd:generateEmbeddedObjectMeta=true rbac:roleName=spark-operator-controller webhook paths="./..." output:crd:artifacts:config=config/crd/bases

.PHONY: generate
generate: controller-gen ## Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
Expand Down
7 changes: 7 additions & 0 deletions api/v1beta2/sparkapplication_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,13 @@ type Dependencies struct {
// SparkPodSpec defines common things that can be customized for a Spark driver or executor pod.
// TODO: investigate if we should use v1.PodSpec and limit what can be set instead.
type SparkPodSpec struct {
// Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
// Spark version >= 3.0.0 is required.
// Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
// +optional
// +kubebuilder:validation:Schemaless
// +kubebuilder:validation:Type:=object
Template *corev1.PodTemplateSpec `json:"template,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will review the rest of the PR as well, but adding this as a struct rather than a binary/string field increases the number of properties in the resulting CRD by quite a bit. Is there any downside/considerations on the K8s side for very large CRDs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I added a kubebuilder marker +kubebuilder:validation:Schemaless to mark the pod template field as schemaless to reduce the size of CRD.

// Cores maps to `spark.driver.cores` or `spark.executor.cores` for the driver and executors, respectively.
// +optional
// +kubebuilder:validation:Minimum=1
Expand Down
5 changes: 5 additions & 0 deletions api/v1beta2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 29 additions & 0 deletions charts/spark-operator-chart/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4745,6 +4745,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -9498,6 +9504,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -10337,6 +10349,23 @@ spec:
May contain labels and annotations that will be copied into the PVC
when creating it. No other fields are allowed and will be rejected during
validation.
properties:
annotations:
additionalProperties:
type: string
type: object
finalizers:
items:
type: string
type: array
labels:
additionalProperties:
type: string
type: object
name:
type: string
namespace:
type: string
type: object
spec:
description: |-
Expand Down
29 changes: 29 additions & 0 deletions charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4684,6 +4684,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -9407,6 +9413,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -10243,6 +10255,23 @@ spec:
May contain labels and annotations that will be copied into the PVC
when creating it. No other fields are allowed and will be rejected during
validation.
properties:
annotations:
additionalProperties:
type: string
type: object
finalizers:
items:
type: string
type: array
labels:
additionalProperties:
type: string
type: object
name:
type: string
namespace:
type: string
type: object
spec:
description: |-
Expand Down
1 change: 1 addition & 0 deletions charts/spark-operator-chart/templates/webhook/mutatingwebhookconfiguration.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ webhooks:
objectSelector:
matchLabels:
sparkoperator.k8s.io/launched-by-spark-operator: "true"
sparkoperator.k8s.io/mutated-by-spark-operator: "true"
rules:
- apiGroups: [""]
apiVersions: ["v1"]
Expand Down
1 change: 1 addition & 0 deletions cmd/operator/webhook/start.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,7 @@ func newCacheOptions() cache.Options {
&corev1.Pod{}: {
Label: labels.SelectorFromSet(labels.Set{
common.LabelLaunchedBySparkOperator: "true",
common.LabelMutatedBySparkOperator: "true",
}),
},
&v1beta2.SparkApplication{}: {},
Expand Down
29 changes: 29 additions & 0 deletions config/crd/bases/sparkoperator.k8s.io_scheduledsparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4745,6 +4745,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -9498,6 +9504,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -10337,6 +10349,23 @@ spec:
May contain labels and annotations that will be copied into the PVC
when creating it. No other fields are allowed and will be rejected during
validation.
properties:
annotations:
additionalProperties:
type: string
type: object
finalizers:
items:
type: string
type: array
labels:
additionalProperties:
type: string
type: object
name:
type: string
namespace:
type: string
type: object
spec:
description: |-
Expand Down
29 changes: 29 additions & 0 deletions config/crd/bases/sparkoperator.k8s.io_sparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4684,6 +4684,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -9407,6 +9413,12 @@ spec:
- name
type: object
type: array
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version >= 3.0.0 is required.
Ref: https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template.
type: object
terminationGracePeriodSeconds:
description: Termination grace period seconds for the pod
format: int64
Expand Down Expand Up @@ -10243,6 +10255,23 @@ spec:
May contain labels and annotations that will be copied into the PVC
when creating it. No other fields are allowed and will be rejected during
validation.
properties:
annotations:
additionalProperties:
type: string
type: object
finalizers:
items:
type: string
type: array
labels:
additionalProperties:
type: string
type: object
name:
type: string
namespace:
type: string
type: object
spec:
description: |-
Expand Down
16 changes: 16 additions & 0 deletions docs/api-docs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2794,6 +2794,22 @@ TODO: investigate if we should use v1.PodSpec and limit what can be set instead.
<tbody>
<tr>
<td>
<code>template</code><br/>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#podtemplatespec-v1-core">
Kubernetes core/v1.PodTemplateSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Spark version &gt;= 3.0.0 is required.
Ref: <a href="https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template">https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template</a>.</p>
</td>
</tr>
<tr>
<td>
<code>cores</code><br/>
<em>
int32
Expand Down
Loading