Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
BIN_DIR=_output/bin
RELEASE_DIR=_output/release
REPO_PATH=volcano.sh/volcano
IMAGE_PREFIX=volcanosh
IMAGE_PREFIX=sailimages
CRD_OPTIONS ?= "crd:crdVersions=v1,generateEmbeddedObjectMeta=true"
CRD_OPTIONS_EXCLUDE_DESCRIPTION=${CRD_OPTIONS}",maxDescLen=0"
CC ?= "gcc"
Expand Down Expand Up @@ -89,9 +89,14 @@ vcctl: init

image_bins: vc-scheduler vc-controller-manager vc-webhook-manager

images:
for name in controller-manager scheduler webhook-manager; do\
docker buildx build -t "${IMAGE_PREFIX}/vc-$$name:$(TAG)" . -f ./installer/dockerfile/$$name/Dockerfile --output=type=${BUILDX_OUTPUT_TYPE} --platform ${DOCKER_PLATFORMS} --build-arg APK_MIRROR=${APK_MIRROR}; \
# build everytime code changes in scheduler
scheduler-image:
docker buildx build -t "${IMAGE_PREFIX}/vc-sail-scheduler:v3" . -f ./installer/dockerfile/scheduler/Dockerfile --output=type=${BUILDX_OUTPUT_TYPE} --platform ${DOCKER_PLATFORMS}

# build only once
images:
for name in controller-manager webhook-manager; do\
docker buildx build -t "${IMAGE_PREFIX}/vc-sail-$$name:v3" . -f ./installer/dockerfile/$$name/Dockerfile --output=type=${BUILDX_OUTPUT_TYPE} --platform ${DOCKER_PLATFORMS}; \
done

generate-code:
Expand Down
46 changes: 12 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

<a href="https://volcano.sh/">
<img src="https://raw.githubusercontent.com/volcano-sh/volcano/master/docs/images/volcano-horizontal-color.png"/>
</a>
Expand All @@ -10,7 +11,6 @@
[![Release](https://img.shields.io/github/release/volcano-sh/volcano.svg)](https://github.com/volcano-sh/volcano/releases)
[![LICENSE](https://img.shields.io/github/license/volcano-sh/volcano.svg)](https://github.com/volcano-sh/volcano/blob/master/LICENSE)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3012/badge)](https://bestpractices.coreinfrastructure.org/projects/3012)
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/volcano-sh/volcano/badge)](https://scorecard.dev/viewer/?uri=github.com/volcano-sh/volcano)


[Volcano](https://volcano.sh/) is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by
Expand Down Expand Up @@ -43,7 +43,7 @@ Volcano is an incubating project of the [Cloud Native Computing Foundation](http

- [Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU](https://sched.co/MPi7)
- [Volcano 在 Kubernetes 中运行高性能作业实践 @ ArchSummit 2019](https://archsummit.infoq.cn/2019/shenzhen/presentation/1817)
- [Volcano:基于云原生的高密计算解决方案 @ Huawei Connection 2019](https://e.huawei.com/cn/material/event/HC/09099dce0070415e9f26ada51b2216d7)
- [Volcano:基于云原生的高密计算解决方案 @ Huawei Connection 2019](https://agenda.events.huawei.com/2019/cn/minisite/agenda.html#dayTab=day7&tagName=%7B%22language%22%3A%22Cn%22%7D&seminarId=1743)
- [Improving Performance of Deep Learning Workloads With Volcano @ KubeCon 2019 NA](https://sched.co/UaZi)
- [Batch Capability of Kubernetes Intro @ KubeCon 2019 NA](https://sched.co/Uajv)
- [Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU](https://sched.co/MPi7)
Expand Down Expand Up @@ -77,7 +77,7 @@ Note:
Install Volcano on an existing Kubernetes cluster. This way is both available for x86_64 and arm64 architecture.

```
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/release-1.10/installer/volcano-development.yaml
kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/release-1.8/installer/volcano-development.yaml
```

Enjoy! Volcano will create the following resources in `volcano-system` namespace.
Expand Down Expand Up @@ -108,24 +108,6 @@ job.batch/volcano-admission-init 1/1 48s 96s

```

### Install via helm

To install official release, please visit to [helm-charts](https://github.com/volcano-sh/helm-charts) for details.

```bash
helm repo add volcano-sh https://volcano-sh.github.io/helm-charts
helm install volcano volcano-sh/volcano -n volcano-system --create-namespace
```

Install from source code for developers:

```bash
helm install volcano installer/helm/chart/volcano --namespace volcano-system --create-namespace

# list helm release
helm list -n volcano-system
```

### Install from code

If you don't have a kubernetes cluster, try one-click install from code base:
Expand All @@ -141,20 +123,18 @@ This way is only available for x86_64 temporarily.
If you want to get prometheus and grafana volcano dashboard after volcano installed, try following commands:

```bash
make TAG=v1.10.0 generate-yaml
kubectl create -f _output/release/volcano-monitoring-v1.10.0.yaml
make TAG=v1.8.2 generate-yaml
kubectl create -f _output/release/volcano-monitoring-v1.8.2.yaml
```

## Kubernetes compatibility

| | Kubernetes 1.17 | Kubernetes 1.18 | Kubernetes 1.19 | Kubernetes 1.20 | Kubernetes 1.21 | Kubernetes 1.22 | Kubernetes 1.23 | Kubernetes 1.24 | Kubernetes 1.25 | Kubernetes 1.26 | Kubernetes 1.27 | Kubernetes 1.28 | Kubernetes 1.29 |Kubernetes 1.30 |
|-----------------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|----------------|
| Volcano v1.6 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - | - | - | - | - | - |- |
| Volcano v1.7 | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - |_ |
| Volcano v1.8 | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - |- |
| Volcano v1.9 | - | - | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |- |
| Volcano v1.10 | - | - | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |✓ |
| Volcano HEAD (master) | - | - | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |✓ |
| | Kubernetes 1.17 | Kubernetes 1.18 | Kubernetes 1.19 | Kubernetes 1.20 | Kubernetes 1.21 | Kubernetes 1.22 | Kubernetes 1.23 | Kubernetes 1.24 | Kubernetes 1.25 | Kubernetes 1.27 | Kubernetes 1.28 |
|------------------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|
| Volcano v1.6 | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | - | - | - | - |
| Volcano v1.7 | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Volcano v1.8 | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |✓ | ✓ | ✓ |
| Volcano HEAD (master) | - | - | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |✓ | ✓ | ✓ |

Key:
* `✓` Volcano and the Kubernetes version are exactly compatible.
Expand All @@ -180,8 +160,6 @@ Resources:

If you have any question, feel free to reach out to us in the following ways:

[Volcano Slack Channel](https://cloud-native.slack.com/archives/C011GJDQS0N) | [Join](https://slack.cncf.io/)
[Volcano Slack Channel](https://volcano-sh.slack.com)

[Mailing List](https://groups.google.com/forum/#!forum/volcano-sh)

Wechat: Add WeChat account `k8s2222` (华为云小助手2号) to let her pull you into the group.
6 changes: 4 additions & 2 deletions installer/helm/chart/volcano/config/volcano-scheduler.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,16 @@ actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: overcommit
arguments:
overcommit-factor: 20
- name: gang
enablePreemptable: false
- name: conformance
- plugins:
- name: overcommit
- name: drf
enablePreemptable: false
- name: predicates
- name: capacity
- name: proportion
- name: nodeorder
- name: binpack
2 changes: 1 addition & 1 deletion installer/helm/chart/volcano/templates/admission.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ spec:
- --port={{.Values.basic.admission_port}}
- -v={{.Values.custom.admission_log_level}}
- 2>&1
image: {{ .Values.basic.image_registry }}/{{.Values.basic.admission_image_name}}:{{.Values.basic.image_tag_version}}
image: {{ .Values.basic.image_registry }}/{{.Values.basic.admission_image_name}}:v3
imagePullPolicy: {{ .Values.basic.image_pull_policy }}
name: admission
{{- if .Values.custom.admission_resources }}
Expand Down
2 changes: 1 addition & 1 deletion installer/helm/chart/volcano/templates/controllers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ spec:
resources:
{{- toYaml .Values.custom.controller_resources | nindent 14 }}
{{- end }}
image: {{ .Values.basic.image_registry }}/{{.Values.basic.controller_image_name}}:{{.Values.basic.image_tag_version}}
image: {{ .Values.basic.image_registry }}/{{.Values.basic.controller_image_name}}:v3
args:
- --logtostderr
- --enable-healthz=true
Expand Down
6 changes: 3 additions & 3 deletions installer/helm/chart/volcano/templates/scheduler.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ rules:
verbs: ["create", "get", "list", "watch", "delete"]
- apiGroups: ["batch.volcano.sh"]
resources: ["jobs"]
verbs: ["get", "list", "watch", "update", "delete"]
verbs: ["get", "list", "watch", "update", "delete", "patch"]
- apiGroups: ["batch.volcano.sh"]
resources: ["jobs/status"]
verbs: ["update", "patch"]
Expand Down Expand Up @@ -169,7 +169,7 @@ spec:
{{- end }}
containers:
- name: {{ .Release.Name }}-scheduler
image: {{ .Values.basic.image_registry }}/{{.Values.basic.scheduler_image_name}}:{{.Values.basic.image_tag_version}}
image: sailimages/vc-sail-scheduler:v3
{{- if .Values.custom.scheduler_resources }}
resources:
{{- toYaml .Values.custom.scheduler_resources | nindent 12 }}
Expand All @@ -188,7 +188,7 @@ spec:
env:
- name: DEBUG_SOCKET_DIR
value: /tmp/klog-socks
imagePullPolicy: {{ .Values.basic.image_pull_policy }}
imagePullPolicy: Never

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be Never? Can you double-check?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the value from the update_1.8 commit at 22809e2#diff-317373f0dc2ee312347cf7e2cce9683f6fc278af96ec2135164c9c0547fb7f06R164.

According to https://kubernetes.io/docs/concepts/containers/images/#image-pull-policy, I believe setting this value to Never tells Kubelet to never fetch a remote image but only uses local ones. This should be intended for our project.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? scheduler is the image that's always updated, right?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we update scheduler-image locally (with docker build) each time we run a submission. After the image is successfully built we run helm upgrade to deploy it.

ImagePullPolicy=Never ensures that during scheduler container creation with helm, we are only looking for a local image (named and tagged sailimages/vc-sail-scheduler:v3) instead of fetching from any remote image registry.

volumeMounts:
- name: scheduler-config
mountPath: /volcano.scheduler
Expand Down
10 changes: 5 additions & 5 deletions installer/helm/chart/volcano/values.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
basic:
controller_image_name: "volcanosh/vc-controller-manager"
scheduler_image_name: "volcanosh/vc-scheduler"
admission_image_name: "volcanosh/vc-webhook-manager"
controller_image_name: "sailimages/vc-sail-controller-manager"
scheduler_image_name: "sailimages/vc-sail-scheduler"
admission_image_name: "sailimages/vc-sail-webhook-manager"
admission_secret_name: "volcano-admission-secret"
admission_config_file: "config/volcano-admission.conf"
scheduler_config_file: "config/volcano-scheduler.conf"
image_pull_secret: ""
image_pull_policy: "Always"
image_tag_version: "v1.10.0"
image_tag_version: "v3"
admission_port: 8443
image_registry: "docker.io"
custom:
Expand Down Expand Up @@ -143,7 +143,7 @@ custom:
webhooks_namespace_selector_expressions: ~


# Specify log level for Volcano main component
# Specify log level for Volcano main component
admission_log_level: 4
scheduler_log_level: 3
controller_log_level: 4
Expand Down
Loading
Loading