KubeCon + CloudNativeCon Japan 2025 Lightning Talk: Optimizing Web Applications by Offloading Heavy Processing To Kubernetes Jobs - Asami Okina
207 Views
July 04, 25
スライド概要
While typical web applications do not require large amounts of resources constantly, there are cases where specific processes consume significant CPU and memory.
In this session, we will introduce an architecture that offloads such resource-intensive processes to Kubernetes Jobs.
We will explain specific methods for Job management, how to integrate web applications (Next.js, @kubernetes/client-node) with the Kubernetes API, methods for data integration between Jobs and web applications, and real-time tracking of Job progress in the UI, all while sharing practical examples. Furthermore, we will provide a detailed introduction to a pattern where Kubernetes Job definitions generated from applications are managed using ConfigMaps, enabling quick configuration switching between environments, and offer hints to optimize your applications in terms of cost, performance, and management.
Session video is available here:
https://youtu.be/8pVt9dhHEVc?feature=shared
Sample code is available here: https://github.com/asami-okina/quick-container-image-scan
KubeCon + CloudNativeCon Japan 2025 speaker. An engineer who develops and operates cloud-native web applications on Kubernetes.
関連スライド
各ページのテキスト
Optimizing Web Applications by Offloading Heavy Processing to Kubernetes Jobs Asami Okina / Craftsman Software, Inc.
Self-Introduction Asami Okina Cloud-Native App Developer Working at CKAD asami-okina asami_okina CKA
Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. The web app is lightweight during normal operations. Needs heavy CPU/memory only during specific operations. Wasting money on peak capacity 24/7 — just for a few moments of actual need.
Solution Offload heavy processing to Kubernetes Jobs Kubernetes standard features only ! Kubernetes Cluster Just what you need, when you need it. Web App Lightweight K8s Job Execute heavy Task Keep the web app lightweight. Need more CPU? Scale with a Job — just in time.
How
Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
Part 1：Job Creation Flow of Creating Jobs from Web App Kubernetes Cluster ① Click Start Process Button Web (UI) Next.js(API) Browser In App pod
Part 1：Job Creation Flow of Creating Jobs from Web App Kubernetes Cluster ① Click Start Process Button ② Job Create Request Web (UI) Next.js(API) Browser In App pod K8s API Server K8s Job Worker pod
Part 1：Job Creation Flow of Creating Jobs from Web App Kubernetes Cluster ① Click Start Process Button ② Job Create Request Web (UI) Next.js(API) Browser In App pod ③ Response HTTP 200 K8s API Server K8s Job Worker pod
Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
Part 2：Job Execution Flow of Job Execution and Real-time Feedback Kubernetes Cluster ① Execute Heavy Processing K8s Job Worker pod
Part 2：Job Execution Flow of Job Execution and Real-time Feedback Kubernetes Cluster ① Execute Heavy Processing K8s Job Worker pod Shared DB ② Write Progress Progress Sharing
Part 2：Job Execution Flow of Job Execution and Real-time Feedback Kubernetes Cluster ① Execute Heavy Processing K8s Job Worker pod Next.js(API) Shared DB ② Write Progress Progress Sharing ③ Poll Progress In App pod
Part 2：Job Execution Flow of Job Execution and Real-time Feedback Kubernetes Cluster ① Execute Heavy Processing K8s Job Worker pod Next.js(API) Shared DB ② Write Progress Progress Sharing ③ Poll Progress In App pod ④ Real-time Display Processing Status File Conversion (75% Complete) About 2 min remaining Web (UI) Browser
Agenda Part 1：Job Creation Part 2：Job Execution Patterns for Creating Jobs from Web App
1 – ConfigMap Job Template Design Pattern Used by the WebApp to create Jobs. After
1 – ConfigMap Job Template Design Pattern Used by the WebApp to create Jobs. Zero App Code Changes No app rebuild or redeployment — just YAML. ConfigMap Inside: Job Template
1 – ConfigMap Job Template Design Pattern Used by the WebApp to create Jobs. Zero App Code Changes No app rebuild or redeployment — just YAML. ConfigMap Inside: Job Template Dynamic env switching Switch resources for dev, test, and prod environments. After Before Add
2 – Dynamic Job Creation Pattern @kubernetes/client-node Next.js(Web app) Official TypeScript Client Kubernetes API
2 – Dynamic Job Creation Pattern 1 Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node.
2 – Dynamic Job Creation Pattern 1 2 Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node. Inject: app-specific values Replace placeholders at runtime using @ctrl/golang-template in Job templates.
2 – Dynamic Job Creation Pattern 1 2 3 Get: Job template from a ConfigMap Retrieve a Job template stored in a ConfigMap using @kubernetes/client-node. Inject: app-specific values Replace placeholders at runtime using @ctrl/golang-template in Job templates. Create: Job Create a Kubernetes Job using @kubernetes/client-node.
Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads.
Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time.
Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time. Points
Wrap-up Problem Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Solution Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time. Points Cost Optimization Resources are allocated only when needed, reducing idle capacity costs.
Wrap-up Problem Solution Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time. Points Cost Optimization Stability Resources are allocated only when needed, reducing idle capacity costs. Built-in retry and parallelization with Kubernetes standard features only.
Wrap-up Problem Solution Always-On Over-Provisioning Excessive CPU/memory allocation for temporary high loads. Offload heavy processing to Kubernetes Jobs. Need more CPU? Scale with a Job — just in time. Points Cost Optimization Stability Operational Efficiency Resources are allocated only when needed, reducing idle capacity costs. Built-in retry and parallelization with Kubernetes standard features only. Flexible configuration using ConfigMaps.
Thank you The potential of Jobs is unlimited! Sample Code https://github.com/asami-okina/ quick-container-image-scan