Machine Learning Engineer II

About the Role

We are looking for a Core Machine Learning Engineer who can take machine learning solutions from prototype to production-grade, scalable systems.

The ideal candidate combines strong fundamentals in machine learning, software engineering, distributed systems, and MLOps. You will work closely with data scientists, software engineers, and product teams to productionize ML models, build reliable ML pipelines, optimize inference and training systems, and scale ML workloads to handle large volumes of real-world traffic and data.

This role is best suited for an engineer who enjoys solving the difficult engineering problems that arise when moving ML from notebooks and experiments into high-performance, reliable production systems.

You should care deeply about our Clients and strive to deliver value through applied science. You should work well in a fast-paced setting, using your skills to create effective solutions. With a combination of deep business know-how and hands-on skills, you will tackle complex challenges and drive continuous improvement.

Key Responsibilities

Productionize ML Solutions

Take ML models and prototypes developed by data scientists/research teams and convert them into production-ready services.

Design robust model serving and inference architectures.

Build APIs and services around ML models for real-time and batch inference.

Develop reusable frameworks and components for deploying and operating ML models.

Establish standards for model versioning, deployment, rollback, monitoring, and lifecycle management.

Identify and eliminate gaps between experimental ML code and production requirements.

Build Scalable ML Systems

Design ML systems capable of handling high throughput, low latency, and large-scale workloads.

Scale model training and inference horizontally across multiple machines/instances.

Optimize CPU, GPU, memory, storage, and network utilization.

Design distributed processing pipelines for large datasets and ML workloads.

Develop asynchronous and batch-processing architectures where appropriate.

Handle bursty workloads and changing traffic patterns efficiently.

ML Infrastructure & MLOps

Build and maintain end-to-end ML pipelines covering:

Data ingestion

Data processing

Feature generation

Model training

Model validation

Model deployment

Model monitoring

Retraining

Implement CI/CD pipelines for ML models and ML services.

Automate model deployment and infrastructure provisioning.

Establish reproducibility across training and production environments.

Implement model and data versioning.

Build monitoring for model performance, infrastructure health, latency, throughput, and data quality.

Model Serving & Inference Optimization

Optimize inference latency and resource consumption.

Evaluate and implement model serving technologies and frameworks.

Apply techniques such as:

* Quantization

* Pruning

* Distillation

* Batching

* Caching

* Parallel inference

* Hardware acceleration

Optimize models for CPU/GPU/accelerator-based environments.

Design intelligent routing and fallback mechanisms for ML inference services.

Establish performance benchmarks and capacity requirements.

Machine Learning Engineering

Implement ML algorithms and production-grade ML pipelines when required.

Understand the underlying mathematical and statistical principles behind ML algorithms.

Work with supervised, unsupervised, and deep learning approaches as appropriate to the problem.

Evaluate model quality using appropriate offline and online metrics.

Investigate model degradation and production performance issues.

Develop mechanisms for continuous model improvement.

Data & Feature Engineering

Build reliable and scalable data pipelines supporting ML workloads.

Design feature engineering pipelines for both training and online inference.

Ensure training-serving consistency.

Handle large-scale structured and unstructured datasets.

Implement data validation and quality checks.

Design efficient feature storage and retrieval mechanisms.

Reliability & Production Engineering

Design ML systems with high availability, fault tolerance, and graceful degradation.

Implement observability across ML systems.

Troubleshoot production issues involving models, infrastructure, data, and services.

Establish SLAs/SLOs for ML services.

Build automated alerting and failure recovery mechanisms.

Participate in production incident response and root-cause analysis.

Collaboration & Technical Leadership

Work closely with Data Scientists, Software Engineers, Data Engineers, DevOps/SRE, and Product teams.

Review ML architecture and production designs.

Establish engineering best practices for ML development.

Drive technical decisions around ML infrastructure, frameworks, and architecture.

Translate experimental ML solutions into maintainable engineering systems.

BASIC QUALIFICATIONS

3+ years of building models for business application experience

Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, Statistics, or a related field.

Strong professional experience in Machine Learning Engineering or a closely related field.

Strong proficiency in Python and experience writing production-quality software.

Strong understanding of machine learning fundamentals.

Strong software engineering fundamentals including:

Data structures and algorithms

Object-oriented design

Design patterns

Distributed systems

Concurrency/parallelism

API design

Testing

Hands-on experience deploying ML models into production.

Experience building scalable ML inference or training systems.

Strong understanding of cloud infrastructure and containerized deployments.

Experience with Docker and Kubernetes or equivalent container orchestration technologies.

Experience with CI/CD and production deployment workflows.

Experience with databases, distributed storage, and caching systems.

Strong debugging and performance optimization skills.

PREFERRED QUALIFICATIONS

Experience with several of the following:

ML Frameworks

PyTorch

TensorFlow

Scikit-learn

XGBoost / LightGBM

Hugging Face Transformers

Model Serving

NVIDIA Triton Inference Server

TorchServe

TensorFlow Serving

ONNX Runtime

vLLM

Ray Serve

FastAPI / gRPC

Distributed Computing

Apache Spark

Ray

Kafka

Distributed task queues

Distributed training frameworks

Cloud & Infrastructure

AWS / GCP / Azure

Kubernetes

Docker

Terraform

GPU infrastructure

Object storage

Managed ML platforms

MLOps

MLflow

Kubeflow

Airflow

Argo

Feature stores

Model registries

Data/model monitoring platforms

Databases & Storage

PostgreSQL / MySQL

Redis

Elasticsearch / OpenSearch

NoSQL databases

Data warehouses/lakes

Performance & Scalability Expectations

The candidate should be comfortable answering questions such as:

How would you take a model serving 10 requests/second to 10,000+ requests/second?

How would you reduce inference latency without materially impacting model quality?

How would you design a highly available model-serving platform?

How would you deploy a new model version without disrupting production traffic?

How would you detect model drift and automatically trigger retraining?

How would you handle training-serving skew?

How would you scale GPU workloads efficiently?

How would you design an ML pipeline capable of processing billions of records?

How would you handle a sudden 10x increase in inference traffic?

How would you debug a model whose offline metrics are good but whose production performance has degraded?

What We Look For

We value candidates who can demonstrate:

Strong ML fundamentals rather than only familiarity with ML libraries.

Strong software engineering discipline and ability to write maintainable production code.

Experience moving ML systems from POC → production → scale.

Ability to reason about latency, throughput, reliability, cost, and model quality together.

Strong understanding of distributed systems and cloud infrastructure.

Ability to profile and optimize ML workloads.

Ownership of systems throughout their lifecycle.

Pragmatic engineering judgment and ability to choose the right technology for a given problem.

Ability to work effectively across research, engineering, infrastructure, and product teams.

Nice to Have

Experience with large-scale recommendation, ranking, search, fraud detection, personalization, or forecasting systems.

Experience working with real-time ML systems.

Experience with GPU optimization and CUDA.

Experience with LLM inference or generative AI systems.

Experience building high-throughput event-driven ML architectures.

Experience implementing online learning or continuous learning systems.

Contributions to open-source ML infrastructure or engineering projects.

Experience designing ML platforms used by multiple teams.