Building Idempotent Callback Systems in Node.js

When we first build APIs, we usually assume every request arrives exactly once.

In reality, distributed systems don't work that way.

Networks fail.

Servers restart.

Clients retry.

Load balancers reconnect.

Robots lose connectivity.

Eventually, the same callback will arrive more than once.

If your application isn't designed for this, a duplicate callback can trigger the same business logic twice—unlocking a roadway twice, finishing the same task twice, or publishing duplicate MQTT commands.

This is where idempotency becomes one of the most important concepts in backend engineering.

What Is Idempotency?

An operation is idempotent if executing it multiple times produces the same final result as executing it once.

Imagine a robot sends this callback:

{
  "reqCode": "REQ-1024",
  "taskCode": "TASK-001",
  "callbackMethod": "finishFE"
}

The first callback finishes the task successfully.

Five seconds later, the robot retries because it never received your HTTP response.

Without idempotency:

Finish task ✔
Unlock roadway ✔
Publish MQTT ✔
 
Retry...
 
Finish task AGAIN ❌
Unlock roadway AGAIN ❌
Publish MQTT AGAIN ❌

Now your entire workflow becomes inconsistent.

Instead, we want:

Finish task ✔
 
Retry...
 
Already processed.
Ignore safely.

No matter how many times the callback is received, the business state remains correct.

Why Duplicate Callbacks Happen

Many engineers assume duplicates are bugs.

Most of the time, they are simply a normal part of distributed systems.

Some common causes include:

HTTP client retries
Network timeouts
Reverse proxy retries
Robot reconnects
Service restarts
Message broker redelivery
Human operators clicking "Retry"

If your service communicates with hardware, PLCs, AGVs, or external APIs, duplicate messages are something you should expect—not something you hope never happens.

Respond Fast, Process Later

One mistake I often see is performing heavy business logic before sending an HTTP response.

For callback-based systems, this increases the chance that the sender retries because it assumes the request failed.

Instead, acknowledge the callback immediately.

exports.callback = (req, res) => {
  res.status(200).json({
    code: "0",
    message: "Success",
    reqCode: req.body.reqCode,
  });
 
  setImmediate(() => {
    processCallbackAsync(req.body).catch(console.error);
  });
};

This pattern gives two major benefits:

The sender receives a response almost instantly.
Business logic can continue safely in the background.

For robot integrations, this significantly reduces unnecessary retries.

Idempotency Is More Than an `if` Statement

A common implementation looks like this:

const exists = await CallbackLog.findOne(query);
 
if (!exists) {
    await processCallback();
}

It appears correct.

Unfortunately, it is not safe.

Imagine two identical callbacks arrive at exactly the same moment.

Callback A
Callback B
 
↓
 
Both execute findOne()
 
↓
 
No document found
 
↓
 
Both continue
 
↓
 
Business logic executed twice

This is called a race condition.

Checking first and then inserting is not atomic.

Using MongoDB as an Atomic Idempotency Guard

A much safer approach is letting MongoDB enforce uniqueness.

const CallbackLogSchema = new mongoose.Schema({
    reqCode: String,
    taskCode: String,
    callbackMethod: String,
    createdAt: {
        type: Date,
        default: Date.now,
        expires: "7d",
    },
});
 
CallbackLogSchema.index(
    {
        reqCode: 1,
        taskCode: 1,
        callbackMethod: 1,
    },
    {
        unique: true,
    }
);

Instead of asking whether the callback already exists, simply try to insert it.

try {
    await CallbackLog.create({
        reqCode,
        taskCode,
        callbackMethod,
    });
} catch (err) {
    if (err.code === 11000) {
        console.log("Duplicate callback ignored.");
        return;
    }
 
    throw err;
}

Because the database guarantees uniqueness, only one request succeeds.

Every duplicate automatically fails with MongoDB's duplicate key error (11000).

This makes the operation atomic and removes race conditions entirely.

A Single Dispatcher Keeps Everything Organized

Once idempotency has been verified, dispatch the callback to the correct handler.

const handler = handlers[callbackMethod];
 
if (!handler) {
    console.warn("Unknown callback.");
    return;
}
 
await handler(body);

Each callback now has its own dedicated function.

Callback
      │
      ▼
Idempotency Guard
      │
      ▼
Dispatcher
      │
      ├── startFE()
      ├── finishFE()
      ├── station()
      ├── updateSeq()
      └── finish()

This architecture is much easier to maintain than hundreds of nested if statements.

Idempotency Doesn't Replace Business Validation

Even after passing the idempotency guard, handlers should still validate their own state.

For example:

await Task.findOneAndUpdate(
    {
        taskCode,
        finishFlag: {
            $ne: true,
        },
    },
    {
        finishFlag: true,
        jobStatus: "finish",
    }
);

Even if another service accidentally invokes the handler directly, the task will only transition to finished once.

Think of this as a second layer of protection.

Why This Matters in Robotics

Unlike traditional web applications, robot integrations interact with the physical world.

A duplicated callback isn't just duplicate data.

It might mean:

Unlocking the same roadway twice
Releasing a station too early
Triggering the same PLC output twice
Publishing duplicate MQTT commands
Moving another robot into an occupied area

These mistakes can lead to production downtime or even safety issues.

Idempotency helps ensure that software behaves predictably, even when communication does not.

Key Takeaways

Distributed systems should assume duplicate requests are normal.
Respond to callbacks immediately to avoid unnecessary retries.
Never rely on findOne() before processing; it is vulnerable to race conditions.
Use database-level unique constraints as your idempotency guard.
Keep business handlers focused on business logic, not duplicate detection.
Add state validation inside handlers as an additional safety layer.
The goal of idempotency is simple: no matter how many times the same callback arrives, the final system state should remain exactly the same.

Reliable systems aren't built by assuming everything works perfectly.

They're built by assuming failures will happen—and designing software that behaves correctly anyway.

References

Martin Kleppmann — Designing Data-Intensive Applications
RFC 9110 — HTTP Semantics (Idempotent Methods)
MongoDB Documentation — Unique Indexes
MongoDB Documentation — TTL Indexes
Microsoft Azure Architecture Center — Idempotent Message Processing
AWS Builder's Library — Making Retries Safe with Idempotent APIs

Building Idempotent Callback Systems in Node.js for Reliable Robot Integrations