Mainframe API errors — 400, 401, 403, 500, 503 and timeouts

What happened

A consumer — a mobile app, web front end or partner — calls a REST API that is served by z/OS Connect in front of CICS, IMS, Db2 or MQ. The call returns an HTTP error or never returns at all.

POST /accounts/v1/transfers
HTTP/1.1 403 Forbidden
Content-Type: application/json
{ "errorMessage": "..." }

client log: request timed out after 30000 ms

What it means

The status code tells you which layer to suspect. A request may pass an API gateway, TLS (AT-TLS or Liberty), z/OS Connect itself, then the back-end system and the program.

Typical causes

• 400: missing field, text in a numeric field, or a value longer than the copybook field.
• 401: expired or wrongly issued token, missing header, or certificate not trusted.
• 403: the authenticated user lacks the API role, or the SAF user ID it maps to lacks access to the transaction or data.
• 500: CICS abend, Db2 negative SQLCODE, or a response that could not be mapped back to JSON.
• 503: API or service stopped in z/OS Connect, back-end connection to CICS or IMS down, or the region is unavailable.
• Timeouts: long-running transaction, Db2 lock waits, slow downstream call, or traffic spikes.

Symptoms

• 4xx errors are usually consistent for one consumer or payload; 5xx and timeouts often affect many consumers at once.
• A 401 or 403 that starts suddenly for everyone often follows a certificate or key change.
• A timeout at the consumer does not mean the work failed — the transaction may have completed after the client gave up.

Where to look

• Gateway logs, using the correlation or request ID.
• z/OS Connect server logs (the Liberty messages log and any FFDC output) for the same time.
• RACF ICH408I messages and SMF records for the mapped user ID.
• CICS region log for abends and DFH messages; Db2 messages for SQL errors or lock timeouts.
• Performance monitoring for response times and transaction volumes.

How to diagnose

1. Collect the exact URL, method, status code, timestamp and correlation ID.
2. Use the table above to pick the layer to check first.
3. For 4xx, reproduce with the same payload and compare it field by field with the API spec.
4. For 403, find the user ID the request ran under and check both the API role and back-end access.
5. For 500 and 503, match the timestamp to back-end logs and to the z/OS Connect logs.
6. For timeouts, compare the consumer, gateway, z/OS Connect and back-end timeout values, then check whether the transaction finished.

How to fix

• 400: correct the consumer payload, or the mapping if the spec was wrong.
• 401/403: renew credentials or request the correct role or access through the security team.
• 500: fix the back-end program or data; redeploy the mapping if it was out of step with the copybook.
• 503: restart the stopped service or connection, or recover the back-end region.
• Timeouts: tune the slow component; set each layer's timeout slightly longer than the one behind it; move genuinely long work to an asynchronous pattern.

How to prevent

• Design update APIs to be idempotent, for example with a client request ID.
• Validate payloads at the gateway against the OpenAPI spec.
• Track certificate and key expiry dates; apply rate limits.
• Regenerate mappings whenever a copybook changes.

Production considerations

Interview question

Consumers report intermittent timeouts on an API that calls a CICS program. How do you investigate?