Senior AI Platform Engineer (Kubernetes & LLMOps)
Senior AI Platform Engineer (Kubernetes & LLMOps)

Job Title:

Senior AI Platform Engineer (Kubernetes & LLMOps)

Category:

Artificial Intelligence (AI) / Machine Learning

Job Number:

3108

Location:

New York, New York, 10036, United States

Temp/Contract Roles:

Direct Hire

Salary:

$0.00 - $210,000.00 210000.00
Job Title: Senior AI Platform Engineer (Kubernetes & LLMOps)
Location: New York, NY

The Mandate We are building the enterprise-grade backbone that powers Generative AI across the firm. We need a Senior Platform Engineer to architect the internal "AI Factory"—the self-service platform that enables our quant, product, and tech teams to deploy GenAI agents and LLMs into production.

This is not a data science role. It is a hard-core engineering role focused on infrastructure, scalability, and developer experience within a highly regulated financial environment. You will bridge the gap between raw Kubernetes infrastructure and high-level AI application logic.

The Tech Stack
  • Core: Python (FastAPI/Flask), Go.

  • Infra: Kubernetes (OpenShift), Helm, Kustomize, Docker.

  • AI/Data: LangChain, LangGraph, Vector Databases, Kafka, Redis.

  • Ops: GitOps (ArgoCD), Jenkins, Prometheus, Grafana, OpenTelemetry.

What You Will Build
  • The AI Platform: Architect reusable, containerized blueprints for deploying GenAI solutions. You will define how the firm deploys Agents, RAG pipelines, and fine-tuned models on OpenShift.

  • Agentic Infrastructure: Design the orchestration layer for multi-agent workflows (using frameworks like LangGraph) to ensure they can run securely and reliably in production.

  • Developer Experience: Abstract away the complexity of K8s and security compliance. Build the tooling and APIs that allow internal teams to spin up AI environments in minutes, not weeks.

  • Enterprise Hardening: Solve for the specific constraints of NYC finance—implementing rigorous OAuth 2.0 flows, secret management, air-gapped model serving, and immutable audit logs.

  • Observability: Implement deep tracing for LLM performance (token usage, latency, drift) using OpenTelemetry and Cortex.

The Candidate Profile
  • Experience: 10+ years in software engineering with 5+ years dedicated to distributed systems or platform engineering.

  • Industry Context: Experience working in Investment Banking, Fintech, or High-Frequency Trading environments. You understand that "speed" cannot come at the expense of "compliance."

  • Kubernetes Mastery: You aren't just a user; you understand the internals of K8s, operators, and mesh networking.

  • GenAI Reality: 1+ years of hands-on work with LLMs. You have moved beyond "Hello World" tutorials and understand the pain points of state management in LangChain or hosting embeddings at scale.

  • Security First: Deep knowledge of RBAC, OIDC, and secure coding practices required for handling sensitive financial data.
     

Why This Role? You won't just be building chatbots; you will be defining the engineering standards for how a major financial institution adopts AI for the next decade.

APPLY NOW

Share this job

Interested in this job?
Save Job

Create Job Alert

Create As Alert

Consultant

Read More
SCHEMA MARKUP ( This text will only show on the editor. )