Shamshir のスタンドアローン版を k8s(minikube) の cronjob に移植した記録です。
cron でバッチを動かす時の問題点
node, ruby, python 他インタプリンタ系の言語でプログラムを作ると nodenv + npm/yarn, rbenv + bundler, pyenv + pip などのエコシステムを使って環境構築することになります。
これを cron で動かそうとすると ~/.nodenv/shims/node などエコシステム上の実行ファイルを crond に教えてあげる必要が出てきます。このため起動方法を bash -l cron.sh など(-l: Make bash act as if it had been invoked as a login shell (see INVOCATION below) 個人の動作環境をシミュレートするような特別な「忖度」が必要になってくる場合も。
このため 自分の環境では動くのに crontab に設定すると動かない! どうして!? となってしまうことが稀によくあります。(^_^;
エコシステムを切り出したコンテナを作ることでこういった問題は解決でき、デプロイ環境が安定します。
もともとコンテナの定義として Content-agnostic, Infrastructure-agnostic という agnostic(= someone who believes that people cannot know whether God exists or not 神様がいるかどうかなんかわかりっこない = 神がいようといまいと関係ない: X に依存しない?) という概念があります。これが、コンテナが Content-agnostic (中身がスクリプトであろうがバイナリであろうが何のプログラミング言語で書かれていようが適切なイメージであれば動く)、Infrastructure-agnostic(物理でも仮想でもLinuxでもMacでも runc や gVisor といった OCI ランタイムがあればそこで差分を吸収して関係なく動く) と呼ばれる所以だと思います。
k8s cronjob はシンプルなバッチ実行環境ですが、プロダクション環境では依存関係などを設定可能な argo などのワークフローに載せることで処理の分割、依存関係の設定、部分実行/再実行がしやすくなるはずです。
k8s cronjob に移植する時の注意点
- minikube でレジストリからイメージを DL せずローカルのイメージを使う場合( imagePullPolicy: Never )、
eval $(minikube docker-env)で minikube の仮想 worker の docker context に切り替えて docker build する必要がある(下記 docker context の切り替えを参照)
docker context の切り替え
1. もともとの docker context は DOCKER ENDPOINT に unix socket 通信を使う
$ docker context ls NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock https://192.168.49.2:8443 (default) swarm $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE thetitle latest 203fc3c16979 25 minutes ago 125MB thetitle v1.0 203fc3c16979 25 minutes ago 125MB kyagi/thetitle 1.0 22d1a9e10b4a 42 minutes ago 125MB node 16-alpine 0e1547c0f4a4 3 weeks ago 110MB gcr.io/k8s-minikube/kicbase v0.0.29 64d09634c60d 2 months ago 1.14GB
2. eval $(minikube docker-env) で切り替えると
$ eval $(minikube docker-env)
3. minikube の仮想 worker の docker context になり DOCKER ENDPOINT が tcp://192.168.x.x になる。minikube にイメージを渡すためにはこちらのコンテキストで docker build する必要がある
$ docker context ls NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default * Current DOCKER_HOST based configuration tcp://192.168.49.2:2376 https://192.168.49.2:8443 (default) swarm Warning: DOCKER_HOST environment variable overrides the active context. To use a context, either set the global --context flag, or unset DOCKER_HOST environment variable. $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE shamshir v1.0 145688e57875 About an hour ago 165MB thetitle v1.0 39a4338f4a4f About an hour ago 125MB <none> <none> da981f50f1eb 2 hours ago 125MB <none> <none> f062c5aecb69 2 hours ago 125MB kyagi/kubia latest d5e0f5c0c6f0 13 days ago 906MB node 16-alpine 0e1547c0f4a4 3 weeks ago 110MB busybox latest ec3f0931a6e6 3 weeks ago 1.24MB node 14-alpine 755b96824e40 4 weeks ago 119MB k8s.gcr.io/kube-apiserver v1.23.1 b6d7abedde39 2 months ago 135MB k8s.gcr.io/kube-proxy v1.23.1 b46c42588d51 2 months ago 112MB k8s.gcr.io/kube-scheduler v1.23.1 71d575efe628 2 months ago 53.5MB k8s.gcr.io/kube-controller-manager v1.23.1 f51846a4fd28 2 months ago 125MB k8s.gcr.io/etcd 3.5.1-0 25f8c7f3da61 4 months ago 293MB k8s.gcr.io/coredns/coredns v1.8.6 a4ca41631cc7 4 months ago 46.8MB k8s.gcr.io/pause 3.6 6270bb605e12 6 months ago 683kB kubernetesui/dashboard v2.3.1 e1482a24335a 8 months ago 220MB kubernetesui/metrics-scraper v1.0.7 7801cfc6d5c0 8 months ago 34.4MB gcr.io/k8s-minikube/storage-provisioner v5 6e38f40d628d 11 months ago 31.5MB
4. ログアウトするともとの context にもどる
いままでとこれから
いままで)
$ cat cron.sh #!/usr/bin/bash set -m shamshir_pat=<HERE_IS_TOKEN> node shamshir-stand-alone.js --owner kyagi --repo awesome-project --label "releasable" --quorum 2 $ crontab -l */15 10-19 * * 1-5 cd /home/ec2-user/git/shamshir/src; bash -l -c /home/ec2-user/git/shamshir/src/cron.sh
これから)
$ kubectl config current-context
minikube
$ kubectl get cronjobs
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
shamshir */15 10-19 * * * False 0 16s 9m49s
$ cat cronjob5.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: shamshir
spec:
schedule: "*/15 10-19 * * *"
jobTemplate:
spec:
backoffLimit: 5
ttlSecondsAfterFinished: 100
template:
spec:
containers:
- name: shamshir
image: shamshir:v1.0
env:
- name: TZ
value: Asia/Tokyo
- name: shamshir_pat
valueFrom:
secretKeyRef:
name: shamshir
key: pat
imagePullPolicy: Never
restartPolicy: OnFailure
parallelism: 1
completions: 1
concurrencyPolicy: "Forbid"
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 5
$ cat secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: shamshir
type: Opaque
stringData:
pat: HERE_IS_TOKEN
$ cat Dockerfile
FROM node:16-alpine
# Create app directory
WORKDIR /app
COPY package*.json ./
RUN npm install
# Bundle app source
COPY . .
WORKDIR /app/src
CMD [ "node", "shamshir-stand-alone.js", "--owner", "kyagi", "--repo", "awesome-project", "--label", "releasable", "--quorum", "2" ]
$ kubectl logs shamshir-27439718-hl7jf
{"level":"info","message":"Shamshir started.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:22"}
{"level":"info","message":"Shamshir got pulls: 2602,2598,2596,2575,2573,2557,2553,2551,2540,2539,2481,2478,2295,2281,1981,1951,1685","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:22"}
{"level":"info","message":"Shamshir added releasable label to pull/2598.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:24"}
{"level":"info","message":"Shamshir added releasable label to pull/2596.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:24"}
{"level":"info","message":"Shamshir added releasable label to pull/2575.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:25"}
{"level":"info","message":"Shamshir removed releasable label from pull/2573.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:26"}
{"level":"info","message":"Shamshir added releasable label to pull/2551.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:28"}
{"level":"info","message":"Shamshir added releasable label to pull/2540.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:29"}
{"level":"info","message":"Shamshir added releasable label to pull/2539.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:30"}
{"level":"info","message":"Shamshir added releasable label to pull/2478.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:31"}
{"level":"info","message":"Shamshir added releasable label to pull/2295.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:32"}
{"level":"info","message":"Shamshir added releasable label to pull/1951.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:34"}
{"level":"info","message":"Shamshir finished.","mode":"live","owner":"kyagi","repo":"awesome-project","service":"shamshir","timestamp":"2022-01-31 00:54:34"}
