ZipStaff Inc.

AI Platform Engineer

Remote•United States•Contract
$62 - $63 hourly
About the Job
Title: AI Platform Engineer
Location: Cary, NC (Remote)
Type: Contract, contract-to-hire

Overview:
ZipStaff is hiring an AI Platform Engineer for our client, a Fortune 10 healthcare and pharmacy leader. You'll build and operate a production agentic AI platform — the layer other engineers build on — and own its reliability, security, and cost. This is a fully remote contract-to-hire role. Candidates must be US citizens or green card holders; sponsorship is not available.

Responsibilities:
  • Build and operate a production agentic AI platform: orchestration, model gateway, agent registry, retrieval, guardrails, evaluation, and observability.
  • Implement production multi-agent orchestration, including state management and retry/fallback design.
  • Operate model serving and gateway capabilities, including routing, fallback, rate limiting, token/usage capture, and serving of open-weight models.
  • Build retrieval-augmented generation, including vector stores, embedding models, chunking, hybrid and semantic search, reranking, and grounding-quality evaluation.
  • Implement agent infrastructure, including Model Context Protocol (MCP), tool/function-calling contracts, agent registries, permission enforcement, and separation of reasoning from acting agents.
  • Design evaluation and LLMOps practices, including eval harnesses, offline and online evaluation, regression suites, drift and hallucination detection, and experiment tracking.
  • Apply guardrails and AI security, including prompt-injection, jailbreak, and data-leakage defense, plus PII detection and redaction in prompts and responses.
  • Instrument observability for LLM and agent calls (token, latency, and cost telemetry) and quality-signal monitoring that feeds alerting.

Required Qualifications:
  • US citizen or green card holder; sponsorship is not available.
  • 8+ years in software or platform engineering, with hands-on production ownership of AI or ML infrastructure.
  • Has built and operated agentic or LLM systems in production, not only prototyped them.
  • LLM and agent orchestration with LangGraph, LangChain, LlamaIndex, CrewAI, AutoGen, or the Google Agent Development Kit (ADK).
  • Model serving and gateway experience (LiteLLM, Portkey, Kong AI Gateway, TrueFoundry, or Vertex AI Model Garden), including routing, fallback, rate limiting, token/usage capture, serving open-weight models (Llama, Mistral, Gemma, Qwen) via vLLM, TGI, Ollama, or Vertex endpoints, and LoRA/QLoRA adapters.
  • RAG experience with Vertex Vector Search, Pinecone, Weaviate, pgvector, or Elasticsearch.
  • Evaluation and LLMOps with RAGAS, DeepEval, Promptfoo, or OpenAI Evals, and experiment tracking with MLflow, LangSmith, Langfuse, Arize Phoenix, or Braintrust.
  • Guardrails and AI security experience with NeMo Guardrails, Guardrails AI, Model Armor, or Lakera.
  • Observability (OpenTelemetry tracing for LLM and agent calls); Kubernetes and containers; GCP preferred (GKE, Vertex AI, Cloud Run); Terraform, CI/CD, and SRE practice for production AI services.
  • Strong Python.

Preferred Qualifications:
  • Go or Java for platform services.

Benefits:
  • Health Insurance.
  • Dental Insurance.
  • Vision Insurance.
  • 401(k) + matching.
  • Paid time off.
  • Referral Program.

About ZipStaff:
ZipStaff connects skilled technology professionals with leading organizations across the United States, matching top talent with impactful contract and full-time opportunities.
To apply, submit your resume through this posting. Qualified candidates will be contacted by a ZipStaff recruiter.