Rafif Dzaky
Open to DevOps roles

Files · ~/notes/postmortems

Incident · PM-001

SEV-3PM-0012026-09-26caught before launch✓ resolved

The résumé page downloaded the résumé instead of showing it

Impact
On machines with a download manager (IDM), every visit to /resume started a PDF download and the viewer stayed blank. Found in review before launch, so no public visitors were affected.
Detection
Reported by me while reviewing the site on my own machine.

Timeline

  1. BuildThe résumé page embedded the PDF so it could be read inline.
  2. ReviewOpening /resume fired a download every time; the embedded viewer showed nothing.
  3. DiagnosisThe embed requests the PDF on page load. Download managers intercept any PDF response, so "view" became "download".
  4. FixThe page now renders the résumé as an image with a text version. The PDF is fetched only when someone clicks Download.

Root cause

The design assumed every browser renders PDFs inline. A page view should never fetch a file that the browser, or a plugin, may treat as a download.

Fix

Image viewer plus a full text version; the PDF is an explicit download link.

What went well

Caught before launch. The text version also made the résumé readable by screen readers and search engines.

Lesson

Test on a real, messy machine, not only a clean browser profile.

Action items

  • Done: Render the résumé as an image and text; no document loads on page view.
  • Done: Make "Download PDF" an explicit, labelled link.

Incident · PM-002

SEV-3PM-0022026-09-26near miss✓ resolved

The Content-Security-Policy would have broken 81 styles

Impact
None in production. Turning on the framework's built-in CSP blocked 81 inline style attributes (layout, charts, colours) in a test run.
Detection
An automated headless-browser test that counts CSP violations on every page before deploy.

Timeline

  1. ChangeEnabled the framework's built-in CSP as part of a security review.
  2. Test81 violations: every style="" attribute was blocked. A style-src-attr directive I added was silently dropped.
  3. DecisionReplaced it with a post-build script that hashes each inline script into the policy.
  4. Verify0 violations on 8 pages; 5 of 5 injected XSS payloads blocked.

Root cause

The built-in policy assumed no inline style attributes, but the site uses them for custom properties and chart widths. The config accepted a directive it then ignored, so the failure was silent.

Fix

Scripts are limited to 'self' plus the SHA-256 of each inline script. Styles allow inline, because CSS can't execute code.

What went well

The test ran before deploy, so the breakage never shipped.

Lesson

A security control needs a test that proves it doesn't break the product, not only a check that it's present.

Action items

  • Done: Build fails if an inline event handler (onclick="") appears.
  • Done: Deploy verifies the CSP is present on the live homepage.

Incident · PM-003

SEV-3PM-0032026-09-27security✓ resolved

Deploy details were visible in public Actions logs

Impact
The repository is public. The deploy workflow hard-coded the deploy host's internal address, user and path, and a connectivity check printed them in the public logs. The address isn't reachable from the internet, and no key or credential was exposed.
Detection
A security review I ran after the site went live.

Timeline

  1. ReviewRead the workflow and its public run logs the way an outsider would.
  2. FindingHost, user and path appeared in plain text, and a verbose check echoed them into every run.
  3. FixMoved them to repository secrets (masked in logs) and made the check quiet.
  4. HardeningPinned third-party actions to commit SHAs and delete the deploy key at the end of every run.

Root cause

"Not a credential" was treated as "fine to publish". Values that are not secrets can still map the inside of a network.

Fix

Secrets for host, user and path; pinned actions; key cleanup; a documented threat model in the repo.

What went well

Credentials were already in secrets, so nothing needed rotating.

Lesson

Review public artifacts (logs, history, docs) as an attacker would, not only the code.

Action items

  • Done: Move host, user and path into repository secrets.
  • Done: Pin third-party actions to commit SHAs.
  • Done: Accept the residual risk: old values stay in git history (a private address; rewriting public history is not worth it).
  • Open: Keep the CI network rule to one host and one port, and re-check it after every change.

Incident · PM-004

SEV-4PM-0042026-09-27near miss✓ resolved

The team radio kept talking after you left the section

Impact
None in production. In testing, jumping to Contact left the radio clip playing, because part of the board was still on screen.
Detection
A scripted test that scrolls and clicks like a visitor, at desktop and mobile widths.

Timeline

  1. TestJumped from the radio board to Contact: audio still playing.
  2. Diagnosis"In view" meant "any part visible", not "what you are looking at".
  3. FixThe radio now belongs to the middle of the screen: it starts in the middle 20% and stops once the board leaves the middle 60%.
  4. VerifyRe-ran the scripted test on desktop and mobile: stops on leave, never overlaps.

Root cause

The trigger measured visibility, not attention.

Fix

Two observers with a gap between them (hysteresis), and a single audio element so two clips can never overlap.

What went well

The test described the behaviour I wanted, so the bug was obvious the first time it ran.

Lesson

Write the test as the user would describe the behaviour, not as the code implements it.

Action items

  • Done: Middle-band start and stop, with hysteresis.
  • Done: Stop on tab hide, on page leave and on mute.

Incident · PM-005

SEV-4PM-0052026-09-27tooling✓ resolved

A stale dev server blocked the dependency install

Impact
Local `npm ci` failed with EPERM before a commit, which cost a few minutes. Nothing shipped broken.
Detection
The install failed loudly.

Timeline

  1. Failure`npm ci` could not replace the framework’s native binary: EPERM.
  2. DiagnosisA preview server from an earlier test was still running and held the file open. Windows locks files in use.
  3. Why it lingeredThe cleanup command assumed Linux (pkill), which does nothing on Windows.
  4. FixStopped the server by PID and reinstalled from the lockfile.

Root cause

The test did not own the lifecycle of the server it started.

Fix

Tests now stop the servers they start; the cleanup uses the platform’s own tools.

What went well

The lockfile made the reinstall deterministic.

Lesson

Clean up what you start, and never assume the Linux toolbox on another OS.

Action items

  • Done: Stop test servers by PID when a test finishes.

recruiter@rafifdzaky: ~

Exec summary · 60 seconds

Rafif Dzaky Daniswara, DevOps Engineer

I own everything after the merge for PT PSM's 20+ production services: the pipelines that ship them, the servers they run on, and the monitoring that tells me when something drifts.

What I own today

  • CI/CD and release approvals
  • Linode VPS fleet: setup, scaling, hardening
  • Domains, DNS and SSL
  • Access, IAM and secrets
  • Monitoring and incident response

Not yet (learning in my homelab)

  • Kubernetes in production
  • AWS or GCP at scale

Evidence

Email [email protected]Contact detailsRésumé