Find Your Next Job
On-Prem Cloud Engineer
Posted on May 9, 2026
- Charlotte, United States of America
- No Salary information.
- Contract
Tailor Your Resume for this Job
Job Duties
Build, configure, and operate on‑prem Kubernetes/OpenShift AI platforms for deploying and serving GenAI models and LLM inference workloads.
· Design and optimize high‑performance inference stacks using vLLM, TensorRT‑LLM, Triton Inference Server, SGLang, and advanced techniques (continuous batching, speculative decoding, KV caching).
· Manage GPU orchestration and capacity using Run:AI, MIG, CUDA/NCCL, and tensor parallelism to maximize utilization and throughput.
· Deploy and operate Kubernetes ML serving frameworks (KServe, Helm, Operators) for scalable, reliable model serving.
· Drive inference optimization and benchmarking, leveraging FP8, AWQ, GPTQ, and performance tools such as GuideLLM and Locust.
· Implement observability and ML monitoring using Prometheus, Grafana, Arize AI, ensuring SLA/SLO compliance for GenAI services.
· Collaborate with ML and research teams to onboard new models, tune inference performance, and productionize
Tech Skills needed
vLLM · TensorRT‑LLM · Triton Inference Server · SGLang · Inference Optimization · Continuous Batching · Speculative Decoding · KV Cache / Prefix Caching · FP8 / AWQ / GPTQ · Tensor Parallelism · Kubernetes ML Serving · KServe · OpenShift AI · Helm / Operators · GPU Orchestration · Run:AI · Performance Benchmarking · CUDA / NCCL / MIG · Prometheus / Grafana · ML Observability GuideLLM, Locust
Pay: From $45.00 per hour
Work Location: In person
Tailor Your Resume for this Job
Share with Friends!
Similar Jobs
Saisir
Backend Python / Devops
L'entreprise & le contexteCette entreprise DeeptTech française en forte croissance d&eac…
Full Time | Neuilly-sur-seine, France
Apply 23 hours, 15 minutes ago
Senior Backend Software Engineer
Role highlights: Senior Backend Software EngineerLocation: Espoo, FinlandDepartment: Fleet Control…
Full Time | Espoo, Finland
Apply 2 days, 23 hours ago
Développeur Principal / Lead Developer
Souhaitez-vous diriger une équipe de développement en télétravail, util…
Full Time | Quebec city, Canada
Apply 6 days, 23 hours ago
Politi
Løsningsarkitekter Til Rigspolitiets Center For International Digitalisering
Vil du være med til at forme de digitale løsninger, der binder politisamarbejdet i Eur…
Full Time | Glostrup, Denmark
Apply 1 week ago
Senior Systems Engineer (Iam)
The Opportunity: Avantor is looking for Senior Identity Access Management (IAM) Engineer will lead …
Full Time | Tn, India
Apply 1 week, 1 day ago
veltris
Srx - Golang Java
Job Information Work Experience 10 - 12 Years Date Opened 04/20/2026 Industry IT Services Job Type …
Full Time | Ka, India
Apply 2 weeks ago
Leyrer + Graf
Cloud Engineer (M/W/D)
Leyrer + Graf Baugesellschaft m.b.H. Gmünd Horn Schwechat Traun Vollzeit GARANTIERTerfolgreich…
Full Time | Gmünd, Austria
Apply 2 weeks, 2 days ago
Senior Software Engineer - Backend
Job Requirements Job Requirements We are building a cloud-native AI platform for medical imaging &…
Full Time | Ka, India
Apply 3 weeks, 2 days ago