Why Does AWS Glue Cost More Than the Job It Runs?

AWS Glue is marketed as serverless, and for anyone who has sized an EC2 fleet or picked an RDS instance class, that sounds like the pricing should be simple: point Glue at a script, let it run, pay for what it did. Then the bill arrives, and the number is bigger than the job felt like it should cost — a nightly ETL job that finishes in four minutes is somehow costing as much as an hour of compute, and a crawler that hasn't found a new file in months is still showing up as a line item every week. Nothing about the console explains why, because there's no server-shaped number anyone chose on purpose. There's a setting, though, and almost nobody revisits it after the job first works.

Why does a "serverless" Glue job have a hardware bill?

Because "serverless" describes who manages the infrastructure, not how billing works. Underneath, every Glue ETL job, crawler, and interactive session runs on workers measured in Data Processing Units (DPUs) — AWS's own unit, where one DPU is a fixed slice of compute: 4 vCPUs and 16 GB of memory. You don't pick a DPU count directly for most job types anymore; you pick a worker type, and the worker type sets the DPUs per worker:

You are then billed for DPU-hours: the number of DPUs allocated, multiplied by however long the job runs, billed per second with a one-minute minimum. Crucially, that's the DPU count you requested, not the DPU count the job actually needed. A job that reads a 40 MB CSV and writes it back out uses a tiny fraction of a G.1X worker's capacity, but if you provisioned five workers, you pay for five workers' worth of DPU-hours regardless.

What does AWS Glue actually cost?

At standard on-demand rates:

Two lines in that table are where most of the surprise lives, and neither is the per-DPU-hour rate itself.

Why is my bill bigger than the job needs?

Job workers are usually over-allocated, and the old default made that easy to inherit. For Spark ETL and streaming jobs run through the legacy MaxCapacity setting, AWS's own API default is 10 DPUs — you can allocate anywhere from 2 to 100, but a job created without an explicit value gets 10 unless someone changes it. A lightweight job that would run comfortably on two G.1X workers, provisioned at that default, pays for five times the DPU-hours it needs, every single run. Multiply that by a job scheduled hourly and the waste compounds quietly for months before anyone notices the worker count was never revisited after the first successful run.

Crawlers bill on a schedule, not on new data. A crawler costs the same $0.44 per DPU-hour whether it finds a thousand new partitions or zero. Pointing a crawler at a bucket on a nightly schedule "just to be safe" means paying for a full crawl every night even during the months nothing in that prefix changes.

The Data Catalog counts more than you'd guess as an "object." AWS defines a billable Data Catalog object as "a table, table version, partition, partition indexes, statistics, database, or catalog" — not just tables. A table partitioned by day and hour accumulates a new partition object every hour it runs, so a handful of tables with fine-grained time partitioning can cross the 1-million-object free tier long before the table count alone would suggest a problem.

How do I find out what's driving the number?

Start with the job's actual resource footprint against its run history:

aws glue get-job-runs --job-name my-etl-job \
  --query 'JobRuns[].{Started:StartedOn,SecondsRun:ExecutionTime,WorkerType:WorkerType,Workers:NumberOfWorkers,State:JobRunState}' \
  --max-results 20 \
  --output table

ExecutionTime is seconds the job actually consumed resources. Multiply Workers by the DPUs-per-worker for that WorkerType (1 for G.1X, 2 for G.2X) and by ExecutionTime ÷ 3600, and you have that run's DPU-hours — the number the rate table above is billed against. A job repeatedly finishing in under two minutes on five or ten workers is a right-sizing candidate before anything else on this page.

For crawlers, runtime alone doesn't tell you whether the crawl found anything — pair it with what it actually changed in the catalog:

aws glue get-crawler-metrics --crawler-name-list my-crawler \
  --query 'CrawlerMetricsList[].{Name:CrawlerName,LastRunSeconds:LastRuntimeSeconds,MedianRunSeconds:MedianRuntimeSeconds,Created:TablesCreated,Updated:TablesUpdated}' \
  --output table

A crawler with a non-trivial MedianRunSeconds and zero TablesCreated/TablesUpdated across recent runs is scanning a source that hasn't changed — every one of those runs is pure DPU-hour spend with nothing to show for it.

How do I bring the cost down without breaking pipelines?

Right-size the worker count against actual ExecutionTime and data volume, not the number that happened to be set the day the job was created. Small jobs processing megabytes rarely need more than two or three G.1X workers.

Move non-urgent jobs to Flex execution. Pre-production runs, backfills, and anything that can tolerate a slower start save roughly a third of the DPU-hour rate for exactly zero code changes — it's a job-configuration flag, not a script rewrite.

Put crawlers on a schedule that matches how often the source actually changes, not "nightly" by default. A source that lands new files weekly doesn't need a nightly crawl; a narrower S3Targets path or an exclude pattern also cuts what each run has to walk.

Enable job bookmarks so incremental jobs process only new data on each run instead of re-reading the entire source — this shrinks ExecutionTime directly, which is the number the DPU-hour bill is multiplied against.

Set an explicit job timeout. A job with no Timeout set defaults to a multi-hour ceiling, so a job that hangs on a bad partition or a stuck connection keeps accruing DPU-hours for hours before AWS stops it, instead of failing fast at a limit that matches how long the job should normally take.

What else should I check while I'm in here?

Glue jobs are rarely the end of the pipeline — they're usually feeding something else, and that something else can be carrying its own waste. A nightly Glue job loading a Redshift warehouse is easy to leave running on schedule even after the dashboards it feeds stop getting opened; see how to cut Redshift costs by pausing idle clusters for how to tell whether the destination is still worth loading at all.

The same question applies when Glue writes into DynamoDB instead of a warehouse. A table provisioned to absorb a nightly bulk-load job's write burst is usually far larger than the table needs the other 23 hours of the day; DynamoDB on-demand vs. provisioned: which is actually cheaper? covers the utilization math that tells you which billing mode fits a bursty ETL write pattern.

More broadly, a per-DPU-hour meter that scales with schedule frequency rather than data volume is exactly the kind of cost that's easy to set once and never look at again — see the wider pass over what a growing AWS pipeline quietly bills for for how it compares to the rest of an account's spend.

How do I catch this before it becomes a real number?

A single over-provisioned job or a crawler running on the wrong schedule looks trivial on its own — a few dollars a month, easy to shrug off. It's the accumulation across every job, crawler, and Data Catalog table added over a year, none of them individually worth a second look, that turns into a line item somebody eventually has to explain.

Connect your AWS account read-only, and Parsivex's daily anomaly checks compare every service's spend — AWS Glue included — against its own trailing baseline, so a genuine step change (a job someone reconfigured with more workers than it needs, a crawler newly scheduled against a much larger prefix) surfaces the next morning instead of at the end of the month. For how those daily checks and severity thresholds work, see cost anomaly alerts, or read how scans work for what a connected account scan reads across the rest of your bill.