Find Your Next Job
On-Prem Cloud Engineer
Posted on May 9, 2026
- Charlotte, United States of America
- No Salary information.
- Contract
Tailor Your Resume for this Job
Job Duties
Build, configure, and operate on‑prem Kubernetes/OpenShift AI platforms for deploying and serving GenAI models and LLM inference workloads.
· Design and optimize high‑performance inference stacks using vLLM, TensorRT‑LLM, Triton Inference Server, SGLang, and advanced techniques (continuous batching, speculative decoding, KV caching).
· Manage GPU orchestration and capacity using Run:AI, MIG, CUDA/NCCL, and tensor parallelism to maximize utilization and throughput.
· Deploy and operate Kubernetes ML serving frameworks (KServe, Helm, Operators) for scalable, reliable model serving.
· Drive inference optimization and benchmarking, leveraging FP8, AWQ, GPTQ, and performance tools such as GuideLLM and Locust.
· Implement observability and ML monitoring using Prometheus, Grafana, Arize AI, ensuring SLA/SLO compliance for GenAI services.
· Collaborate with ML and research teams to onboard new models, tune inference performance, and productionize
Tech Skills needed
vLLM · TensorRT‑LLM · Triton Inference Server · SGLang · Inference Optimization · Continuous Batching · Speculative Decoding · KV Cache / Prefix Caching · FP8 / AWQ / GPTQ · Tensor Parallelism · Kubernetes ML Serving · KServe · OpenShift AI · Helm / Operators · GPU Orchestration · Run:AI · Performance Benchmarking · CUDA / NCCL / MIG · Prometheus / Grafana · ML Observability GuideLLM, Locust
Pay: From $45.00 per hour
Work Location: In person
Tailor Your Resume for this Job
Share with Friends!
Similar Jobs
Senior Software Engineer - Backend
Job Requirements Job Requirements We are building a cloud-native AI platform for medical imaging &…
Full Time | Ka, India
Apply 3 days, 7 hours ago
Senior Cloud Admin
Our Company At Teradata, we believe that people thrive when empowered with better information. Tera…
Full Time | Mh, India
Apply 1 week ago
Mehiläinen
Detection Engineer, Mehiläinen
Founded in 1909, Mehiläinen is a rapidly growing healthcare company that strongly invests in t…
Full Time | Helsinki, Finland
Apply 1 week, 1 day ago
Private Cloud Engineers – Din Ekspertise Styrker Danmarks Sikkerhed
Forsvarets Efterretningstjeneste (FE) beskytter Danmarks sikkerhed ved at indsamle, analysere og de…
Full Time | København, Denmark
Apply 1 week, 3 days ago
Principal Network Engineer
Aviso: At Aviso, we are dedicated to improving the financial well-being of Canadians. As a leading …
Full Time | Toronto, Canada
Apply 1 week, 3 days ago
Cyber Engineer
Job #: 10735 Division: Information Technology Services Affiliation: Non-Union: Management & Exe…
Full Time | Toronto, Canada
Apply 1 week, 6 days ago
Data Pipeline Engineering Specialist (Cybersecurity) (Hybrid)
We’re seeking someone to join our team as a Data Pipeline Engineering Specialist to design, b…
Full Time | Montréal, Canada
Apply 1 week, 6 days ago
Devsecops Engineer
GE Vernova is seeking an experienced DevSecOps Engineer to support the successful design, configura…
Full Time | Perth, Australia
Apply 2 weeks, 1 day ago