Case study: the Okta support-system breach and session tokens in HAR files (2023)

Okta's support engineers can ask customers for a HAR file, a recording of the browser's HTTP traffic, to reproduce a problem, and customers attach them to cases in Okta's customer support system. Okta said its production service was not affected. On 20 October 2023 Okta disclosed that an attacker had used stolen credentials to reach that system and view files uploaded by some customers, and warned that HAR files can contain cookies and session tokens.

Okta's root cause analysis of 3 November 2023 gave the details. From 28 September to 17 October the attacker had access to files belonging to 134 customers, under 1% of Okta's customer base. Some files were HAR files whose session tokens the attacker used to hijack the legitimate Okta sessions of five customers. Okta revoked the session tokens embedded in the stolen files on 17 and 19 October.

The attacker logged in as a service account that could view and update customer support cases. During the investigation Okta found that an employee had signed in to a personal Google profile in Chrome on an Okta-managed laptop, and that the service account's username and password had been saved into that personal Google account. Okta concluded that the most likely route to the credential was a compromise of the employee's personal Google account or personal device.

Two weaknesses combined. A shared service account authenticated with a password that a person could type, save and sync, and the managed laptop allowed a personal browser profile that synced saved passwords to an account the company did not control. Okta's fixes followed: it disabled the service account and configured Chrome Enterprise so employees cannot sign in to Chrome on managed laptops with a personal Google profile. Chrome's BrowserSignin and RestrictSigninToPattern enterprise policies control which accounts can sign in to the browser.

For your own systems, give integrations workload identity (/domains/identity-crypto/identity/workload-identity-models/) or vault-issued, short-lived credentials instead of passwords, keep any remaining service-account secrets in a secret manager that people cannot browse, and make sure no human-usable credential for a privileged system can end up in a synced browser profile.

A HAR (HTTP Archive) file records every request and response the browser made while recording: URLs with their query strings, request and response headers, cookies, form fields and response bodies. That includes the Cookie header carrying the session ID, Authorization: Bearer tokens, OAuth authorization codes in callback URLs, and tokens returned in JSON. A valid session cookie stands in for the user: it does not need the password, and the MFA prompt already happened before the cookie was issued.

Browsers have improved. Since Chrome 130, DevTools exports HAR files without the Cookie, Set-Cookie and Authorization headers by default, and including them needs an explicit setting. Tokens in URLs, request bodies and responses are still exported. After the breach Cloudflare published an open-source HAR sanitizer (github.com/cloudflare/har-sanitizer), and Okta now tells customers to sanitize credentials and session tokens before sharing a HAR file.

The script below removes the headers and cookies that carry credentials, redacts query parameters, form fields and JSON keys whose names suggest secrets, replaces anything shaped like a JWT, and drops base64-encoded bodies it cannot inspect. It was tested on Python 3.14 against a HAR containing a session cookie, a Bearer JWT, an OAuth code, an ID token and a refresh token in a JSON response, and a password in a form post. Treat its output as reduced risk, not zero: review the file, revoke the sessions you recorded, and have support systems sanitize uploads on arrival.

"""Strip credentials from a HAR file before it leaves your machine: python sanitize_har.py in.har out.har"""
import json
import re
import sys
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit

DROP_HEADERS = {"authorization", "proxy-authorization", "cookie", "set-cookie", "x-api-key"}
SECRET_NAME = re.compile(r"(?i)token|code|secret|passw|session|sid|key|assertion|signature|auth")
JWT = re.compile(r"eyJ[\w-]{4,}\.[\w-]{4,}\.[\w-]*")  # header.payload.signature
GONE = "[REDACTED]"


def scrub(value):
    """Redact secret-looking fields in JSON, and JWTs in any text."""
    if isinstance(value, dict):
        return {k: GONE if SECRET_NAME.search(k) else scrub(v) for k, v in value.items()}
    if isinstance(value, list):
        return [scrub(v) for v in value]
    return JWT.sub(GONE, value) if isinstance(value, str) else value


def scrub_pairs(pairs):
    return [(k, GONE if SECRET_NAME.search(k) else scrub(v)) for k, v in pairs]


def scrub_text(text, mime=""):
    if "x-www-form-urlencoded" in mime:
        return urlencode(scrub_pairs(parse_qsl(text, keep_blank_values=True)), safe="[]")
    try:
        return json.dumps(scrub(json.loads(text)))
    except ValueError:
        return scrub(text)


def scrub_message(message):
    message["headers"] = [h for h in message.get("headers", []) if h["name"].lower() not in DROP_HEADERS]
    message["cookies"] = []  # the parsed copy of the Cookie and Set-Cookie headers
    for field in ("queryString", "params"):
        for pair in message.get(field, []) + message.get("postData", {}).get(field, []):
            pair["value"] = GONE if SECRET_NAME.search(pair["name"]) else scrub(pair["value"])
    if "url" in message:
        parts = urlsplit(message["url"])
        query = scrub_pairs(parse_qsl(parts.query, keep_blank_values=True))
        message["url"] = urlunsplit(parts._replace(query=urlencode(query, safe="[]")))
    for body in (message.get("postData"), message.get("content")):
        if body and body.get("encoding") == "base64":
            body["text"] = ""  # binary or encoded bodies cannot be inspected: drop them
        elif body and body.get("text"):
            body["text"] = scrub_text(body["text"], body.get("mimeType", ""))


if __name__ == "__main__":
    har = json.load(open(sys.argv[1], encoding="utf-8"))
    for entry in har["log"]["entries"]:
        scrub_message(entry["request"])
        scrub_message(entry["response"])
    json.dump(har, open(sys.argv[2], "w", encoding="utf-8"), indent=2)

Okta's investigation started on 29 September, after 1Password's report, but for 14 days it found no suspicious downloads. The reason was in the logs: opening a file attached to a support case produced one log event, while going straight to the support system's Files tab, as the attacker did, produced a different event type with a different record ID. The early searches looked for case access. Only the IP address BeyondTrust supplied on 13 October led Okta to the service account and its activity.

Customers detected the attack before the provider did. 1Password and BeyondTrust reported suspicious activity in their own Okta instances, BeyondTrust supplied the IP address that unlocked the investigation, and Cloudflare detected activity on 18 October, more than 24 hours before Okta notified it. Detection that depends on one expected access path is blind to the others: list every way data can be read or exported, make each one produce an alertable event, and alert on a service account downloading files it has never touched.

Phishing-resistant MFA protects the login, not the session that follows it. Once a valid session token is copied, the controls that matter are those the server applies on every request: binding the session to the client, keeping it short, and requiring fresh authentication for sensitive actions.

Okta's response shows the pattern. It released session token binding based on network location, so an administrator must re-authenticate when a session is reused from a network with a different autonomous system number (ASN), and set new Admin Console defaults of a 12-hour session and a 15-minute idle timeout, the NIST AAL3 limits. Cloudflare's advice to Okta customers was to use hardware MFA, shorten session lifetimes and investigate unexpected MFA and support-initiated changes. Device Bound Session Credentials, which bind cookies to a key in the device's TPM, address the same theft for browser sessions; the session guide at /domains/identity-crypto/authentication/session-lifecycle/ covers it with timeout values.

Cloudflare contained the October attack, then rotated credentials it believed had been exposed. On 1 February 2024 it disclosed that an attacker had been in its systems from 14 to 24 November 2023 using credentials stolen in the Okta compromise: Cloudflare had failed to rotate one service token and three service account credentials, out of thousands, because it mistakenly believed they were unused. They gave access to its Atlassian systems and source code repositories.

Its response was thorough: it rotated more than 5,000 production credentials, triaged 4,893 systems, reimaged and rebooted every machine in its global network, and deleted the HAR files on its own wiki. The lesson is that after a provider's breach you rotate every credential the provider held or could have seen, including the ones you think are dead, and you keep a record of each rotation; see /domains/identity-crypto/secrets/leaked-secret-response/ for the order.

• Treat HAR files, logs, crash dumps and screen recordings as credentials: sanitize before upload, sanitize again on arrival, restrict who can read them and delete them when the case closes.
• Never ask a customer for a HAR file without telling them how to sanitize it, and revoke the sessions it recorded after use.
• Give service accounts non-human credentials, and block personal browser profiles and password sync on managed devices.
• Bind administrator sessions to the client, keep them short, and require reauthentication for sensitive actions.
• Make every data-access path produce an alertable log event, and test detection against the paths an attacker would choose.
• After any breach of a provider, rotate everything it could have exposed, including credentials believed unused.

Sources & further reading

The standards, specifications and documentation behind the guidance above. Each is the evidence for a claim on this page; consult it for detail beyond the defaults and versions listed here.

1. Okta: tracking unauthorized access to Okta's support system (20 October 2023)
2. Okta: unauthorized access to Okta's support case management system, root cause and remediation (3 November 2023)
3. Okta customer support incident update, 29 November 2023 (SEC Form 8-K exhibit 99.2)
4. Cloudflare: how Cloudflare mitigated yet another Okta compromise (20 October 2023)
5. Cloudflare: introducing HAR Sanitizer (26 October 2023)
6. Cloudflare: Thanksgiving 2023 security incident (1 February 2024)
7. 1Password: Okta support system incident and 1Password
8. Chrome for Developers: What's new in DevTools (Chrome 130), HAR exports exclude sensitive data

Reference information

Tags

Account takeoverAuthentication and sessionsWorkload identityIncident response

SDLC phase

DevelopmentMaintenance

Role

DevelopersPlatform engineers and SREs

ASVS 5.0.0 requirements

v5.0.0-7.4.1 · v5.0.0-7.4.2 · v5.0.0-7.5.1 · v5.0.0-16.2.1

CAPEC attack patterns

• CAPEC-593 at MITRE
• CAPEC-60 at MITRE
• CAPEC-560 at MITRE

Download this guide as Markdown

Continue exploring