Title: AI Platform Engineer
Location: Cary, NC (Remote)
Type: Contract, contract-to-hire
Overview:
ZipStaff is hiring an AI Platform Engineer for our client, a Fortune 10 healthcare and pharmacy leader. You'll build and operate a production agentic AI platform — the layer other engineers build on — and own its reliability, security, and cost. This is a fully remote contract-to-hire role. Candidates must be US citizens or green card holders; sponsorship is not available.
Responsibilities:
- Build and operate a production agentic AI platform: orchestration, model gateway, agent registry, retrieval, guardrails, evaluation, and observability.
- Implement production multi-agent orchestration, including state management and retry/fallback design.
- Operate model serving and gateway capabilities, including routing, fallback, rate limiting, token/usage capture, and serving of open-weight models.
- Build retrieval-augmented generation, including vector stores, embedding models, chunking, hybrid and semantic search, reranking, and grounding-quality evaluation.
- Implement agent infrastructure, including Model Context Protocol (MCP), tool/function-calling contracts, agent registries, permission enforcement, and separation of reasoning from acting agents.
- Design evaluation and LLMOps practices, including eval harnesses, offline and online evaluation, regression suites, drift and hallucination detection, and experiment tracking.
- Apply guardrails and AI security, including prompt-injection, jailbreak, and data-leakage defense, plus PII detection and redaction in prompts and responses.
- Instrument observability for LLM and agent calls (token, latency, and cost telemetry) and quality-signal monitoring that feeds alerting.
Required Qualifications:
- US citizen or green card holder; sponsorship is not available.
- 8+ years in software or platform engineering, with hands-on production ownership of AI or ML infrastructure.
- Has built and operated agentic or LLM systems in production, not only prototyped them.
- LLM and agent orchestration with LangGraph, LangChain, LlamaIndex, CrewAI, AutoGen, or the Google Agent Development Kit (ADK).
- Model serving and gateway experience (LiteLLM, Portkey, Kong AI Gateway, TrueFoundry, or Vertex AI Model Garden), including routing, fallback, rate limiting, token/usage capture, serving open-weight models (Llama, Mistral, Gemma, Qwen) via vLLM, TGI, Ollama, or Vertex endpoints, and LoRA/QLoRA adapters.
- RAG experience with Vertex Vector Search, Pinecone, Weaviate, pgvector, or Elasticsearch.
- Evaluation and LLMOps with RAGAS, DeepEval, Promptfoo, or OpenAI Evals, and experiment tracking with MLflow, LangSmith, Langfuse, Arize Phoenix, or Braintrust.
- Guardrails and AI security experience with NeMo Guardrails, Guardrails AI, Model Armor, or Lakera.
- Observability (OpenTelemetry tracing for LLM and agent calls); Kubernetes and containers; GCP preferred (GKE, Vertex AI, Cloud Run); Terraform, CI/CD, and SRE practice for production AI services.
- Strong Python.
Preferred Qualifications:
- Go or Java for platform services.
Benefits:
- Health Insurance.
- Dental Insurance.
- Vision Insurance.
- 401(k) + matching.
- Paid time off.
- Referral Program.
About ZipStaff:
ZipStaff connects skilled technology professionals with leading organizations across the United States, matching top talent with impactful contract and full-time opportunities.
To apply, submit your resume through this posting. Qualified candidates will be contacted by a ZipStaff recruiter.