Skip to content
notes.vinhnq.com

Search

System DesignSenior22 min read

Integrating a VNPay-style payment gateway: redirect, signatures and IPN

Review 14 cards
On this page

Payment gateway integration series: 1. Redirect, signatures and IPN · 2. Late IPNs, reconciliation and failure modes

Most payment bugs aren’t in the call to the gateway’s API but in what happens around it: the customer closes the tab halfway, the gateway calls back three times, the amount is off by 100× because someone forgot the unit. This post walks the full VNPay-style redirect flow (MoMo, ZaloPay, OnePay and Stripe Checkout share the pattern) along the course diagram, steps 1 to 9.2.3: the data model, URL signing, the IPN, the return URL and security. Part 2 covers operations: IPNs that arrive after inventory was released, reconciliation, sandbox testing and a failure-mode table.

Read first: Idempotency part 2 (a minimal IPN handler) and the encoding, hashing and signatures post (HMAC basics). The example is the booking from the airline booking database series, and it applies unchanged to orders. Choosing a gateway (the order of security, reliability, performance, complexity; criteria, fees, the current PCI DSS) and timeouts, retries, circuit breakers and webhooks are in the Third-party integration series.

The vnp_* parameters follow VNPay’s integration docs, version 2.1.0. Specs change; always check the current docs the gateway gives you when you register as a merchant. The signing code was run with JDK 21, the SQL on PostgreSQL 16.

1. The parties and the flow from 1 to 9.2.3

Key idea: seven parties take part, but the backend trusts only two things: the amount it computed itself (steps 2–4) and the signed IPN (step 9.1). Anything that goes through the browser can be edited or may never arrive.

sequenceDiagram
    actor U as Customer
    participant FE as Frontend
    participant BE as Backend
    participant DB as Database
    participant PF as Gateway page (PGW FE)
    participant PB as Gateway backend (PGW BE)
    participant BK as Bank
    U->>FE: 1. Book
    FE->>BE: 2. Checkout
    BE->>DB: 3. Save booking PENDING_PAYMENT and payment PENDING
    BE-->>FE: 4. Signed payment URL
    FE->>PF: 5. Redirect to the gateway page
    U->>PF: 6. Enter card or wallet details
    PF->>BK: 7. Authenticate (OTP)
    BK-->>PB: 8. Charge result
    PB->>BE: 9.1.1 Server-to-server IPN
    BE->>DB: 9.1.2 Verify, update status
    BE-->>PB: reply code
    PB-->>PF: 9.2.1 Result
    PF->>FE: 9.2.2 Redirect to the return URL
    FE->>BE: 9.2.3 Ask for the booking status
    BE-->>FE: status from the DB
Step Who How far to trust it
2–4 The backend computes the amount, creates records, signs the URL The amount is computed by the backend, never taken from the client
5–8 The customer and the gateway, outside our control The customer can drop out anywhere
9.1 The gateway calls our backend directly (IPN) The source of truth: signed, and resent by the gateway if it doesn’t get a valid reply
9.2 The browser returns to the return URL Display only; parameters can be edited, and it may never arrive

Paths 9.1 and 9.2 run in parallel with no ordering. The return URL can arrive a few seconds before the IPN. So at step 9.2.3 the frontend asks the backend and accepts a “processing” state, instead of inferring the result from URL parameters.

The money flow is not the information flow

Key idea: “paid” (a successful IPN, day T) and “money received” (settlement, T+1) are two different events. Revenue reports and the ledger must keep them apart.

  • Day T: money moves from the customer’s account into the gateway’s account at the customer’s bank; the gateway reports the result to the backend (the IPN).
  • Day T+1: the gateway settles (settlement: the actual transfer, minus fees) into the merchant’s bank account.
  • The gateway reconciles with both banks along the way (part 2).
flowchart LR
    K[Customer account] -->|"Day T: transfer"| G[Gateway account at customer's bank]
    G -->|reports result| P[Payment gateway]
    P -->|IPN| M[Merchant backend]
    P -->|"Day T+1: settlement minus fees"| A[Merchant bank account]

2. The data model: booking and payment attempt

Key idea: don’t bolt a payment_status column onto the booking. A booking has several payment attempts, and each attempt is its own payment row with its own reference.

For example: the card is declined and the customer tries a wallet, or the URL expires and they click pay again. The gateway_callback table stores every callback verbatim for debugging and reconciliation.

CREATE TABLE payment (
  payment_id     bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  booking_id     bigint      NOT NULL REFERENCES booking,
  txn_ref        text        NOT NULL UNIQUE,          -- vnp_TxnRef: one per attempt
  gateway        text        NOT NULL DEFAULT 'VNPAY',
  amount         bigint      NOT NULL CHECK (amount > 0),   -- VND, not multiplied by 100
  status         text        NOT NULL DEFAULT 'PENDING'
                 CHECK (status IN ('PENDING', 'SUCCEEDED', 'FAILED', 'EXPIRED', 'REFUND_PENDING', 'REFUNDED')),
  pay_url        text        NOT NULL,
  expires_at     timestamptz NOT NULL,                 -- matches vnp_ExpireDate
  gateway_txn_no text,                                 -- vnp_TransactionNo
  response_code  text,
  bank_code      text,
  paid_at        timestamptz,
  created_at     timestamptz NOT NULL DEFAULT now(),
  updated_at     timestamptz NOT NULL DEFAULT now(),
  UNIQUE (gateway, gateway_txn_no)
);
-- At most one open attempt per booking
CREATE UNIQUE INDEX payment_one_pending_idx ON payment (booking_id) WHERE status = 'PENDING';
CREATE INDEX payment_pending_expiry_idx ON payment (expires_at) WHERE status = 'PENDING';

-- Store every callback verbatim before processing: for debugging and reconciliation
CREATE TABLE gateway_callback (
  callback_id  bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  gateway      text        NOT NULL,
  channel      text        NOT NULL CHECK (channel IN ('IPN', 'RETURN', 'QUERY')),
  txn_ref      text,
  payload      jsonb       NOT NULL,
  signature_ok boolean     NOT NULL,
  reply_code   text,
  received_at  timestamptz NOT NULL DEFAULT now()
);

Two state machines (a list of states and the allowed transitions) run side by side. The booking:

stateDiagram-v2
    [*] --> PENDING_PAYMENT
    PENDING_PAYMENT --> PAID: payment SUCCEEDED
    PENDING_PAYMENT --> EXPIRED: expired, no open payment left
    PENDING_PAYMENT --> CANCELLED: customer cancels
    PAID --> TICKETED: tickets issued
    TICKETED --> CANCELLED: cancel and refund

The payment (one attempt):

stateDiagram-v2
    [*] --> PENDING
    PENDING --> SUCCEEDED: IPN or query reports success
    PENDING --> FAILED: IPN or query reports failure
    PENDING --> EXPIRED: past deadline, query confirms unpaid
    EXPIRED --> SUCCEEDED: late IPN
    SUCCEEDED --> REFUND_PENDING: booking can no longer be honoured
    REFUND_PENDING --> REFUNDED: gateway confirms refund

Note the EXPIRED → SUCCEEDED edge: if real money was taken, it must be recorded, even though we had given up on the attempt. That’s the source of the late-IPN case study in part 2.

Splitting the tables also splits the code: Payment is its own bounded context (a business area with its own model), and Booking only talks to it through a port. A package skeleton for the Booking / Flight / Payment split is in DDD and package structure.

3. Step 4: building the signed payment URL

Key idea: the backend hands the frontend a URL to the gateway page, carrying an HMAC-SHA512 signature computed with a hash secret only we and the gateway know. The gateway recomputes it, so anyone who edits the amount breaks the signature.

An HMAC is an authentication code computed from the content and a secret key: without the key you can’t produce a valid one. How to sign:

  1. Sort the parameters by name (except vnp_SecureHash), drop empty ones.
  2. Join name=url_encoded_value pairs with &.
  3. Compute HMAC-SHA512 with the hash secret, output hex.

Three parameters people get wrong: vnp_Amount is the amount times 100; vnp_TxnRef must not repeat; vnp_CreateDate/vnp_ExpireDate are yyyyMMddHHmmss in Vietnam time (GMT+7).

final class Signer {
    private final byte[] secret;
    Signer(String secret) { this.secret = secret.getBytes(StandardCharsets.UTF_8); }

    static String canonical(SortedMap<String, String> params) {
        return params.entrySet().stream()
                .filter(e -> e.getValue() != null && !e.getValue().isEmpty())
                .map(e -> e.getKey() + "=" + URLEncoder.encode(e.getValue(), StandardCharsets.US_ASCII))
                .collect(Collectors.joining("&"));
    }

    String sign(SortedMap<String, String> params) {
        try {
            Mac mac = Mac.getInstance("HmacSHA512");
            mac.init(new SecretKeySpec(secret, "HmacSHA512"));
            return HexFormat.of().formatHex(mac.doFinal(canonical(params).getBytes(StandardCharsets.US_ASCII)));
        } catch (GeneralSecurityException e) {
            throw new IllegalStateException(e);
        }
    }

    boolean verify(Map<String, String> received) {
        SortedMap<String, String> fields = new TreeMap<>(received);
        String given = fields.remove("vnp_SecureHash");
        fields.remove("vnp_SecureHashType");
        if (given == null) return false;
        byte[] expected = sign(fields).getBytes(StandardCharsets.US_ASCII);
        return MessageDigest.isEqual(expected, given.toLowerCase(Locale.ROOT).getBytes(StandardCharsets.US_ASCII));
    }
}
The vnp_* parameter table, buildPayUrl and test results
Parameter Meaning Note
vnp_Version, vnp_Command 2.1.0, pay
vnp_TmnCode Merchant code Issued at registration
vnp_Amount Amount times 100 6,820,000 VND is sent as 682000000
vnp_CurrCode VND
vnp_TxnRef Reference of this attempt Must not repeat
vnp_OrderInfo Order description No diacritics, avoid special characters
vnp_ReturnUrl Where the browser comes back From server config, never from the client
vnp_IpAddr Customer’s IP The real IP behind the reverse proxy
vnp_CreateDate, vnp_ExpireDate yyyyMMddHHmmss in Vietnam time (GMT+7) The URL’s validity window
vnp_SecureHash Signature Computed over all the other parameters
static final DateTimeFormatter VNP_TIME =
        DateTimeFormatter.ofPattern("yyyyMMddHHmmss").withZone(ZoneId.of("Asia/Ho_Chi_Minh"));

String buildPayUrl(Payment p, String clientIp, Instant now) {
    SortedMap<String, String> q = new TreeMap<>();
    q.put("vnp_Version", "2.1.0");
    q.put("vnp_Command", "pay");
    q.put("vnp_TmnCode", cfg.tmnCode());
    q.put("vnp_Amount", Long.toString(Math.multiplyExact(p.amount(), 100L)));  // the gateway's unit
    q.put("vnp_CurrCode", "VND");
    q.put("vnp_TxnRef", p.txnRef());
    q.put("vnp_OrderInfo", "Thanh toan booking " + p.bookingCode());
    q.put("vnp_OrderType", "other");
    q.put("vnp_Locale", "vn");
    q.put("vnp_ReturnUrl", cfg.returnUrl());
    q.put("vnp_IpAddr", clientIp);
    q.put("vnp_CreateDate", VNP_TIME.format(now));
    q.put("vnp_ExpireDate", VNP_TIME.format(p.expiresAt()));
    String query = q.entrySet().stream()
            .map(e -> URLEncoder.encode(e.getKey(), StandardCharsets.US_ASCII) + "="
                    + URLEncoder.encode(e.getValue(), StandardCharsets.US_ASCII))
            .collect(Collectors.joining("&"));
    return cfg.payUrl() + "?" + query + "&vnp_SecureHash=" + signer.sign(q);
}

The full version run with JDK 21 (fake secret):

  • The URL contains vnp_Amount=682000000 and vnp_CreateDate=20261118100000 for 03:00 UTC.
  • verify returns true for the original parameters, false after changing vnp_Amount to 100, and still true when the signature is upper-cased.
  • Math.multiplyExact throws ArithmeticException on overflow.

More rules for this step:

  • The amount comes from the DB, not the request. The frontend sends only the booking_code.
  • The hash secret lives in a secret manager (Vault, AWS Secrets Manager), not in application.yml or the repo, with a rotation procedure.
  • vnp_ExpireDate is shorter than the seat hold: the URL expires after 15 minutes and seats are held for 20, so a customer paying at minute 14 still has a seat (part 2).
  • The customer’s IP comes from X-Forwarded-For only behind a reverse proxy you trust, taking the address your own proxy appended, not the first element the client can set.

4. Step 9.1: the IPN handler

Key idea: the IPN (Instant Payment Notification) is a signed GET from the gateway’s servers to the registered IPN URL. The handler checks in a fixed order, changes state idempotently in one transaction, and replies with an RspCode that tells the gateway whether to resend.

The IPN URL is configured on the merchant side, not sent per transaction like the return URL. The request carries vnp_TxnRef, vnp_Amount, vnp_ResponseCode, vnp_TransactionStatus, vnp_TransactionNo, vnp_BankCode, vnp_PayDate, vnp_SecureHash. The backend replies with JSON {"RspCode": "..", "Message": ".."}.

RspCode When What the gateway does
00 Result recorded (success or failure) Stops sending
01 No order for vnp_TxnRef Logs a merchant-side error
02 Order already confirmed Stops sending
04 Amount mismatch Logs an error
97 Invalid signature Logs an error
99 Other error (DB down, exception) Resends later

Order of checks: store verbatim → signature → order → amount → status → update.

@RestController
@RequiredArgsConstructor
class VnpayIpnController {
    private final Signer signer;
    private final CallbackLog callbackLog;
    private final PaymentResultService results;

    @GetMapping("/payments/vnpay/ipn")
    IpnReply ipn(@RequestParam Map<String, String> params) {
        boolean signatureOk = signer.verify(params);
        long logId = callbackLog.save("IPN", params, signatureOk);            // its own transaction, always stored
        IpnReply reply;
        try {
            reply = !signatureOk ? IpnReply.INVALID_SIGNATURE
                                 : results.apply(GatewayResult.fromVnpay(params));
        } catch (Exception e) {
            log.error("IPN failed txnRef={}", params.get("vnp_TxnRef"), e);
            reply = IpnReply.UNKNOWN_ERROR;                                   // 99: let the gateway resend
        }
        callbackLog.reply(logId, reply.rspCode());
        return reply;
    }
}

record IpnReply(@JsonProperty("RspCode") String rspCode, @JsonProperty("Message") String message) {
    static final IpnReply CONFIRMED = new IpnReply("00", "Confirm Success");
    static final IpnReply ORDER_NOT_FOUND = new IpnReply("01", "Order not found");
    static final IpnReply ALREADY_CONFIRMED = new IpnReply("02", "Order already confirmed");
    static final IpnReply INVALID_AMOUNT = new IpnReply("04", "Invalid amount");
    static final IpnReply INVALID_SIGNATURE = new IpnReply("97", "Invalid signature");
    static final IpnReply UNKNOWN_ERROR = new IpnReply("99", "Unknown error");
}

PaymentResultService.apply is a shared function for the IPN and the transaction-query job (part 2). It is idempotent (calling it again gives the same result as calling it once) thanks to the status condition in WHERE, and runs in one transaction:

-- 1. Move the payment, only from an unsettled status (EXPIRED is accepted: real money was taken)
UPDATE payment
SET    status = 'SUCCEEDED', gateway_txn_no = :txnNo, response_code = '00',
       bank_code = :bank, paid_at = :payDate, updated_at = now()
WHERE  txn_ref = :txnRef AND status IN ('PENDING', 'EXPIRED')
RETURNING booking_id;
-- 0 rows: already processed → reply 02 (or 00), do nothing else

-- 2. Move the booking, only from PENDING_PAYMENT, with version and history (see the booking/ledger post)
UPDATE booking
SET    status = 'PAID', paid_at = :payDate, version = version + 1, updated_at = now()
WHERE  booking_id = :bookingId AND status = 'PENDING_PAYMENT';

-- 3a. 1 row: seats HELD → BOOKED, write the ledger, insert an "issue tickets" outbox row
-- 3b. 0 rows: the booking expired or was cancelled → late IPN (part 2)
  • Success requires both vnp_ResponseCode = 00 and vnp_TransactionStatus = 00.
  • A failed result is recorded too (payment FAILED) and answered with 00, so the gateway stops resending.
Storing callbacks in a separate or shared transaction, and handler test results

Tested: the first successful IPN moves the payment to SUCCEEDED and the booking to PAID at version 2; the same IPN sent again updates 0 rows in both statements; creating a second PENDING attempt for the same booking is rejected by the partial unique index.

5. Step 9.2: the return URL and the result page

Key idea: the return page only displays the real status fetched from the backend; nothing is written to the DB from this path.

  • Read vnp_TxnRef to know which booking, call GET /bookings/{code}, display it.
  • Still PENDING_PAYMENT (IPN not here yet): show “confirming your payment”, poll with backoff (1s, 2s, 4s), then send the customer to “my bookings”.
  • You can check the return parameters’ signature to show an early message.

The backend can treat the browser’s return as a signal to query that attempt’s transaction early (part 2) instead of waiting for the periodic job. The result still goes through apply.

6. HTTPS and security

Key idea: the course slide just says “HTTPS”; in practice you also need signatures on everything you receive, replay protection from the state machine, and card data that never touches your systems.

  • HTTPS on every hop: the return URL, the IPN URL, the query API. An IPN over plain HTTP lets a man in the middle read and replay it.
  • Verify signatures on everything you receive, with a constant-time comparison (MessageDigest.isEqual: always compares every byte) so the signature doesn’t leak through response timing.
  • Signatures don’t prevent replay: a valid IPN sent again still has a valid signature. Replay protection is the job of the state machine and unique constraints.
  • Never touch card data: see the block below.
  • The return URL is fixed in config, never taken from the client (no open redirect, i.e. being abused to send customers to a fake page).
  • Mask logs: never log the hash secret; IPN parameters can be logged (no card numbers), but mask emails and phone numbers if present.
  • Filtering IPN source IPs against the gateway’s published list (if any) is an extra layer, not a replacement for the signature.

Exercises

Exercise 1: Review an IPN handler

Task: a handler receives the return URL; if vnp_ResponseCode == "00" it marks the booking PAID by vnp_TxnRef with UPDATE booking SET status = 'PAID' WHERE booking_code = ?, then sends a confirmation email, then redirects to a thank-you page. List the bugs and fix them.

Solution to exercise 1: eight bugs
  1. The return URL is used as the source of truth: move to the IPN; the return URL only displays.
  2. No signature check: verify before anything else.
  3. No amount check (nor the ×100 unit): compare vnp_Amount / 100 with payment.amount.
  4. vnp_TransactionStatus isn’t checked: check both codes.
  5. The UPDATE has no status condition: an EXPIRED or CANCELLED booking gets overwritten to PAID; add AND status = 'PENDING_PAYMENT' and handle the 0-row branch.
  6. The booking code is the txn_ref: no retries possible; split out a payment table.
  7. The email is sent synchronously before commit: move it to an outbox.
  8. No ledger, no history, no stored callback: add all three in the same transaction (the callback stored separately).

Exercise 2: A fixed test for the signature

Task: write a test that guarantees any change to the signing code is caught, including switching encoding libraries.

Solution to exercise 2: a fixture-based test

Take a parameter set and signature known to be correct (from the docs or a real sandbox transaction, saved as a fixture) and check three things:

  • sign(fixture) produces exactly the expected signature.
  • verify returns true for the fixture and false when any single character of any parameter changes.
  • A value with spaces and special characters (/, :, &) signs exactly as the gateway does.

Run it in CI; it’s cheap and catches “invalid checksum” bugs before they reach production.

Remember

  • Explain why the IPN is the source of truth and the return URL is display only, and why the two paths have no ordering.
  • Sketch the payment table split from booking, both state machines, and the EXPIRED → SUCCEEDED edge.
  • Name the three signing steps and three common causes of “invalid checksum”.
  • Recite the IPN handler’s check order and what 00, 01, 02, 04, 97, 99 mean.
  • Know which work must leave the IPN handler (outbox) and why.

Interview questions

Junior

What is an IPN and how does it differ from the return URL?

The IPN is a signed server-to-server callback and the source of truth; the return URL is the browser coming back, for display only.

An IPN (Instant Payment Notification) goes straight from the gateway to our backend and is resent until the gateway gets a valid reply. The return URL’s parameters can be edited, and it may never arrive if the customer closes the tab.

Why must the backend compute the amount rather than take it from the frontend?

Because the frontend is in the user’s hands: editing a request to pay 1,000 VND for a 6 million ticket takes seconds.

The backend computes the amount from the booking in the DB and signs it into the URL. The gateway checks the signature so the amount can’t change in transit, and on the IPN the backend compares the amount with the DB again.

Mid

What does the HMAC signature on the payment URL protect, and what doesn’t it?

It protects integrity and origin; it doesn’t stop replay, doesn’t hide content, and is useless once the secret leaks.

Without the hash secret nobody can change the amount, order id or return URL, and a forged IPN won’t have a valid signature. Replaying a valid message needs the state machine and unique constraints; secrecy needs HTTPS; a leaked secret needs a secret manager and rotation.

In what order should an IPN handler check things, and which codes does it reply with?

Store raw → signature (97) → order (01) → amount (04) → already settled (02) → state change (00); system errors reply 99 so the gateway resends.

Compare the amount in the right unit (divide by 100). Failed results are recorded too and answered with 00 so the gateway stops sending.

Why keep a payment table separate from the booking?

Because one booking has several payment attempts, and each attempt needs its own reference.

A declined card, an expired URL or a different method each create a new attempt; the gateway won’t accept a repeated reference, and each callback must match exactly one attempt. A separate table gives a clear history, allows a partial unique “at most one open attempt”, and handles two attempts both succeeding.

Senior

What do you say when asked to send the confirmation email right inside the IPN handler?

Don’t: write an outbox row in the same transaction as the state change, and let a worker send the email after commit.

Email can’t join a DB transaction: send before commit and you email about orders that roll back; send after commit and a crash in between loses the email. It also slows the handler, so the gateway times out and resends. The worker sends idempotently per booking; the IPN handler only changes state and replies fast.

A customer reports “I paid but the site says it failed”. How do you investigate?

Start from gateway_callback by txn_ref, ask the gateway through the query API in parallel, and check the frontend last.

  • Did the IPN arrive, was the signature valid, what did we reply.
  • No IPN: check the IPN URL registered with the gateway and the reverse proxy logs. Replied 97: suspect an encoding mismatch in signing. Replied 04: suspect the unit conversion.
  • Call the transaction-query API for the gateway’s truth and run apply if it says paid.
  • Frontend: does the return page poll the backend when the IPN is late, or conclude from URL parameters?

Useful phrases for interviews

  • “We treat the IPN as the source of truth; the return URL only brings the user back and triggers a status check.”
  • “The amount is computed on the backend from the booking and signed into the URL, never taken from the client.”
  • “Each payment attempt gets its own transaction reference, because the gateway won’t accept a repeated one.”
  • “Most invalid-checksum errors come from building a different canonical string, usually a plus versus %20 for spaces.”
  • “The unit conversion to the gateway’s ×100 amount lives in exactly one place.”
  • “The IPN handler stores the raw callback first, then verifies, then runs one idempotent state transition.”
  • “A signature proves who sent the message, not that it’s the first time; replay protection comes from the state machine.”
  • “Using the redirect flow keeps card data off our servers and our PCI DSS scope minimal.”

Notes based on the Advanced Backend course (Ronin Engineer), rewritten in my own words.