Queues and background jobs

cloudhost queues create emails-dead
cloudhost queues create emails --dead-letter-queue emails-dead --max-retries 5
# created queue emails
cloudhost queues ls

A queue holds messages, each a JSON value up to 128 KB, until a consumer takes them. The dead-letter queue is another queue of the same team. Defaults: a 30-second visibility timeout, three retries (after 5, 10 and 20 seconds) and retention up to the plan's maximum. Delivery is at least once; order is best effort, not guaranteed.

[[queues.producers]]
binding = "JOBS"
queue   = "emails"

delay_seconds holds delivery back (up to an hour on Hobby, 7 days on Pro). dedup_id stops repeats: the same id within 10 minutes is accepted once, the others are counted in duplicates. A batch is up to 100 messages and 256 KB. Sending with the API needs a key with the developer role or higher. The older [[queue]] shape with binding and name is still accepted.

# in the consumer's project file: on deploy this function becomes the queue's consumer
[[queues.consumers]]
queue             = "emails"
max_batch_size    = 10
max_batch_wait_ms = 2000
max_retries       = 5
dead_letter_queue = "emails-dead"

Returning normally acks the whole batch; throwing retries the messages not acked. batch.ackAll() and batch.retryAll() exist too. A message can arrive again, so the consumer must tolerate repeats.

If the consumer has not answered by the visibility timeout, the batch returns to the queue and is delivered again. A function can consume several queues; batch.queue names the one. A queue has one consumer: a function, a URL or pull.

cloudhost queues consumer emails --url https://example.com/hooks/emails --concurrency 4
# signing secret (shown once): whsec_…
cloudhost queues consumer rotate-secret emails   # a new secret; the old one stops signing

The platform POSTs each message on its own to your HTTPS address; the body is the message's body as JSON. The headers follow Standard Webhooks, so any Standard Webhooks library verifies the signature:

• webhook-id: the message id, to drop repeats.
• webhook-timestamp: when it was sent, in Unix seconds.
• webhook-signature: v1,<base64>, an HMAC-SHA256 over id.timestamp.body with the queue's whsec_ secret.
• queue-name and queue-attempt: the queue and this attempt's number.

A 2xx answer acks the message. Any other answer, or none by the visibility timeout, retries it with backoff. The signing secret is shown once; keep it in the receiving app's secrets. Private and internal addresses are refused.

cloudhost queues consumer emails --pull
cloudhost queues pull emails --max 10 --visibility-timeout 60
cloudhost queues ack emails <lease_id> <lease_id>
cloudhost queues ack emails <lease_id> --retry --delay 30

For apps outside the platform: take up to 100 messages, process them, and ack each by its lease_id. A message neither acked nor retried by the visibility timeout returns to the queue. Pull works only on a queue whose consumer is pull (not_pull).

After each failure the message comes back after a delay that starts at retry_delay_s (default 5 seconds) and doubles each time (at most 12 hours). When max_retries (up to 100) are used up it moves to the dead-letter queue, with the queue it came from and why. Without a dead-letter queue it is dropped.

cloudhost queues dead-letters emails                  # browse what failed
cloudhost queues dead-letters retry emails --limit 100 # back into emails, attempts reset
cloudhost queues retry emails <id> <id>                # only these
cloudhost queues dead-letters purge emails             # delete them
cloudhost queues redrive emails-dead                   # from the dead-letter queue: each back where it came from

The dead-letter queue is an ordinary queue: browse, retry or purge it. Fix the cause, then retry; the messages start again with fresh retries. Purging needs a second-factor step-up.

cloudhost schedules create "0 3 * * *" --queue emails --name nightly --body '{"task":"digest"}'
cloudhost schedules create "*/15 * * * *" --url https://example.com/hooks/tick --tz UTC
cloudhost schedules ls
cloudhost schedules pause <id>
cloudhost schedules rm <id>

A schedule sends a message on a cron expression (five fields) in a time zone (default Asia/Tehran): to a queue, a function or a URL. URLs are signed as in URL delivery; a URL schedule's signing secret is shown only when it is created. last_at and last_status show the last run. Hobby has 10 schedules, Pro 1,000.

cloudhost queues stats emails --range 24h
#   backlog 12 (ready 9, delayed 1, in flight 2), oldest 4s, 2 KB
#   per minute: sent 40, delivered 39, acked 38, failed 1, dead 0
cloudhost queues messages emails --state in_flight
cloudhost queues pause emails      # sends still accepted
cloudhost queues resume emails

GET /v1/queues/<name>/stats gives the ready, delayed and in-flight counts, the stored size, the oldest message's age and per-minute rates over the last 5 minutes; …/observability gives the charts over 1 hour, 24 hours or 7 days. In the dashboard: Queues, with a chart, the consumer, a peek at the messages (never edited), dead letters, schedules and settings.

An operation is one send, one delivery, or one ack or retry, of up to 64 KB; a larger message counts one per 64 KB. Price: 80,000 tomans per million operations, from the rial wallet. On Hobby, after 10,000 operations a day requests are refused with 429 ops_limit until the next day; a full backlog (queue_full) refuses new sends too. Queue operations are their own line on the invoice.