Cloud ML platforms

What a managed platform actually bundles

A cloud ML platform like SageMaker or Vertex AI isn’t one product — it’s a bundle of services that would otherwise be separate things you’d have to provision and wire together yourself: managed training jobs that handle GPU allocation and distributed setup, a model registry for versioning trained artifacts, one-click or API-driven deployment to an autoscaling serving endpoint, built-in experiment tracking, and monitoring for drift and performance once a model is live. The appeal is that these pieces are pre-integrated. Submit a training job and the platform handles spinning up the right instance type, mounting the data, running the job, and tearing the compute down when it finishes — work that otherwise requires writing and maintaining IaC definitions, orchestration DAGs, and serving infrastructure by hand, as the two previous lessons in this group cover. The platform is, in effect, someone else’s already-built and already-operated version of all of that.

The tradeoff is speed and safety versus cost and control

The honest framing of build-vs-buy here isn’t “managed platforms are for small teams and custom infrastructure is for big ones” — that’s too simple, and plenty of large, sophisticated ML orgs run heavily on managed platforms. The real tradeoff is along two axes. Speed and operational safety: a managed platform gets a team from trained model to a working, autoscaled, monitored endpoint in days, run by a small team, with the platform vendor responsible for the serving infrastructure staying up. Cost and control: that convenience carries a real premium over raw compute pricing, and it caps what you can customize — if your workload needs an unusual distributed training topology, custom hardware the platform doesn’t support, or serving latency characteristics the managed endpoint can’t hit, you’re fighting the platform’s abstractions instead of being helped by them.

A concrete way to decide

A useful test: can you describe your training and serving requirements using the platform’s supported job types, instance types, and deployment patterns without working around them? If yes, the platform likely saves real engineering time, because you’re paying for infrastructure work you would otherwise have to build and, more importantly, operate — keep patched, keep scaled, keep monitored — indefinitely. If your requirements routinely fall outside what the platform supports — custom accelerators, nonstandard parallelism strategies, extremely latency-sensitive serving that needs infrastructure control the platform abstracts away — building on raw infrastructure (using the IaC and orchestration tools from the earlier lessons in this group) gives you that control, at the cost of now owning the operational burden the platform would otherwise have carried. Team size matters here mostly as a proxy for whether you have the platform engineering capacity to own that burden well, not as a rule on its own.

Lock-in is real, but not where people usually point

The common worry is “if we build on SageMaker, we’re stuck with AWS forever.” The sharper version of that worry is more useful: what’s actually locked in is the operational workflow — pipeline definitions expressed in the platform’s SDK, deployment configurations tied to its specific APIs, monitoring dashboards built on its specific tooling. A trained model itself is usually a standard artifact (an ONNX file, a PyTorch checkpoint, a container image) that can, in principle, run somewhere else with some rework. Migrating off a platform means rebuilding the surrounding workflow, not retraining the model from scratch. That’s still a real cost — often underestimated — but it’s a different, smaller cost than the all-or-nothing framing suggests, and it’s worth evaluating against honestly rather than treating lock-in as an automatic disqualifier.

Why this matters going forward

Cloud ML platforms are the point where the infrastructure-as-code and orchestration disciplines from the earlier lessons meet a genuine business decision: build the equivalent yourself, with full control and full operational responsibility, or rent an integrated version of it and accept its boundaries. Neither answer is correct in general — it depends on whether your workload fits inside what the platform supports and whether your team would rather spend its time on infrastructure or on the model itself. The next lesson, on distributed training infrastructure, is exactly the kind of requirement that can push that decision one way or the other: some distributed training setups are well inside what managed platforms handle cleanly, and some aren’t.