Skip to content

NagaYu/AdSentry

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ›‘οΈ AdSentry

Real-time anomaly detection for your ad spend. Because your AI bidder doesn't sleep β€” and neither should your guardrails.

Catch CPA spikes and runaway "Advantage+/PMax" campaigns the moment they start burning money β€” and get a loud, mention-tagged alert in Slack & Discord before the damage is done.

License: MIT TypeScript Node Status


πŸ”₯ Why AdSentry exists

In 2018, a "campaign" was a human-tuned machine. You set the bids. You watched the dashboard. If something broke, you broke it, and you knew exactly what you'd touched.

In the AI era, that's gone. Advantage+ Shopping, Performance Max, automated bidding, and Smart Campaigns are black boxes that spend your money on your behalf at machine speed. When they work, they're magic. When they fail β€” a broken pixel, a busted feed, a bad creative, an algorithm chasing a phantom signal β€” they can torch a month's budget between two coffees, and the dashboard won't tell you until tomorrow's report.

Marketing teams have monitoring for everything except the thing that actually spends the money.

SRE figured this out a decade ago. You don't stare at a Grafana dashboard waiting for a server to fall over β€” you set SLOs, define error budgets, and let alerting page you when reality drifts from expectation. AdSentry brings that exact discipline to ad spend.

AdSentry is the "Sentry" (as in sentinel, guard) for AdOps:

  • It polls Meta Ads & Google Ads on a schedule (default: hourly).
  • It computes your live CPA and compares it to a trailing 7-day baseline.
  • It detects two failure modes that scream "the AI has gone rogue":
    • CPA Spike β€” you're paying far more per conversion than you historically should.
    • Zero-Conversion Burn β€” spend keeps climbing while conversions flatline at zero.
  • It alerts β€” instantly, loudly, with <!here> / @here mentions, in Slack and Discord.

No dashboards to babysit. No "I wish someone had caught this sooner." Just a sentry on the wall.


✨ Features

πŸ”Œ Multi-platform Meta Graph API and Google Ads API out of the box.
πŸ“ˆ Baseline-aware detection Compares live CPA against a trailing 7-day average, not a naive static threshold.
🧠 Runaway-AI heuristics Catches both CPA spikes and the zero-conversion-burn signature.
🎨 Rich, loud alerts Slack Block Kit + Discord Rich Embeds, color-coded by severity, with @here mentions.
πŸ›‘οΈ Production-grade resilience Exponential backoff + jitter, full 429 / Retry-After handling, per-request timeouts.
♻️ 24/7 daemon node-cron scheduler, run-lock against overlapping ticks, clean SIGINT/SIGTERM shutdown.
πŸ§ͺ Simulation mode Exercise the entire alert path β€” including a synthetic "runaway" campaign β€” with zero credentials.
πŸ”’ Strict TypeScript strict: true, noUncheckedIndexedAccess, exactOptionalPropertyTypes. The compiler is on your side.

πŸ—ΊοΈ How it works

flowchart TD
    Cron["⏰ node-cron scheduler<br/>(default: hourly)"] --> Fetch

    subgraph Fetch["πŸ“₯ Fetcher"]
        direction TB
        Meta["Meta Graph API<br/>/insights"]
        Google["Google Ads API<br/>googleAds:search (GAQL)"]
        Retry{"429 / 5xx /<br/>network error?"}
        Meta --> Retry
        Google --> Retry
        Retry -->|yes, retries left| Backoff["Exponential backoff<br/>+ jitter, honor Retry-After"]
        Backoff --> Retry
        Retry -->|success| Normalize["Normalize β†’ PerformanceSnapshot[]<br/>+ 7-day BaselineStats"]
    end

    Normalize --> Detect

    subgraph Detect["🧠 Detector"]
        direction TB
        CPA["Compute live CPA = Cost Γ· CV"]
        Compare{"CV == 0 &&<br/>Cost > waste cap?"}
        Spike{"CPA > baseline Γ—<br/>spike threshold?"}
        CPA --> Compare
        Compare -->|yes| Anomaly["🚨 ZERO_CONVERSION"]
        Compare -->|no| Spike
        Spike -->|yes| Anomaly2["🚨 HIGH_CPA"]
        Spike -->|no| Healthy["βœ… Healthy"]
    end

    Anomaly --> Notify
    Anomaly2 --> Notify
    Healthy --> Quiet["😴 No alert<br/>(optional heartbeat)"]

    subgraph Notify["πŸ“£ Notifier"]
        direction TB
        Slack["Slack Block Kit<br/>(red attachment + &lt;!here&gt;)"]
        Discord["Discord Rich Embed<br/>(red embed + @here)"]
    end

    Notify --> Humans["πŸ‘©β€πŸ’» On-call AdOps<br/>fixes it in minutes, not days"]
Loading

The data contract in one line: fetcher β†’ PerformanceSnapshot[] β†’ detector β†’ AnomalyResult[] β†’ notifier β†’ Slack/Discord.


⚑ 10-second quick start

# 1. Install
git clone https://github.com/adsentry/adsentry.git
cd adsentry
npm install

# 2. Build
npm run build

# 3. See it fire β€” no credentials needed (synthetic "runaway" campaign included)
node dist/index.js check --simulate --heartbeat

You'll watch AdSentry evaluate four synthetic campaigns, flag the two deliberately-broken ones (a Meta CPA spike and a Google zero-conversion burn), and print the exact math behind each verdict. Point a webhook at it and those same alerts land in Slack/Discord.


πŸš€ Real usage

One-shot scan (great for cron / CI)

node dist/index.js check
# Exits 1 if any anomaly was found, 0 if all clear β€” wire it into anything.

Resident 24/7 watcher

node dist/index.js watch
# Runs an immediate scan, then re-scans every hour. Ctrl-C to stop cleanly.

# Custom schedule (every 15 minutes) + heartbeat on healthy cycles:
node dist/index.js watch --cron "*/15 * * * *" --heartbeat

Legacy flag forms adsentry --check and adsentry --watch are also supported.


πŸ”§ Configuration

All configuration is via environment variables (a .env file is auto-loaded via dotenv).

Detection thresholds

Variable Default Meaning
CPA_SPIKE_PERCENT 150 Fire HIGH_CPA when live CPA exceeds this % of the 7-day baseline.
ZERO_CONVERSION_COST_CAP 50 Fire ZERO_CONVERSION when CV == 0 and spend exceeds this (account currency).
ABSOLUTE_CPA_CAP (unset) Optional hard ceiling β€” fire if live CPA exceeds this regardless of baseline.
MIN_COST_FOR_EVALUATION 5 Ignore tiny windows below this spend (noise suppression).
MIN_BASELINE_CONVERSIONS 3 Require this many baseline conversions before trusting the baseline CPA.
LOOKBACK_DAYS 7 Trailing baseline window length.

Scheduling & resilience

Variable Default Meaning
CRON_EXPRESSION 0 * * * * 5-field cron schedule for watch mode.
MAX_RETRIES 5 Max retry attempts per API/webhook request.
RETRY_BASE_DELAY_MS 1000 Base delay for exponential backoff.
RETRY_MAX_DELAY_MS 30000 Backoff ceiling.
REQUEST_TIMEOUT_MS 20000 Per-request timeout.
SIMULATE false Force synthetic data mode globally.

Meta Ads credentials

Variable Required Example
META_ACCESS_TOKEN βœ… EAAB... (a long-lived system-user token)
META_AD_ACCOUNT_ID βœ… act_1234567890 (prefix added automatically if omitted)
META_API_VERSION β€” v19.0

Google Ads credentials

Variable Required Example
GOOGLE_ADS_DEVELOPER_TOKEN βœ… abcd1234...
GOOGLE_ADS_CLIENT_ID βœ… ...apps.googleusercontent.com
GOOGLE_ADS_CLIENT_SECRET βœ… GOCSPX-...
GOOGLE_ADS_REFRESH_TOKEN βœ… 1//0g...
GOOGLE_ADS_CUSTOMER_ID βœ… 1234567890 (dashes stripped automatically)
GOOGLE_ADS_LOGIN_CUSTOMER_ID β€” MCC id, if accessing via a manager account
GOOGLE_ADS_API_VERSION β€” v17

Configure either platform, both, or neither (with --simulate). AdSentry skips any platform whose credentials are incomplete, and if all live fetches fail it falls back to simulation so you always get a heartbeat instead of silent darkness.

Example .env

# --- Alerting ---
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T000/B000/XXXX
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/000/XXXX
SLACK_MENTION=<!here>
DISCORD_MENTION=@here

# --- Detection ---
CPA_SPIKE_PERCENT=150
ZERO_CONVERSION_COST_CAP=50

# --- Meta ---
META_ACCESS_TOKEN=EAAB...
META_AD_ACCOUNT_ID=act_1234567890

# --- Google Ads ---
GOOGLE_ADS_DEVELOPER_TOKEN=...
GOOGLE_ADS_CLIENT_ID=...apps.googleusercontent.com
GOOGLE_ADS_CLIENT_SECRET=GOCSPX-...
GOOGLE_ADS_REFRESH_TOKEN=1//0g...
GOOGLE_ADS_CUSTOMER_ID=1234567890

πŸ’¬ Setting up Slack alerts (2 minutes)

  1. Go to https://api.slack.com/apps β†’ Create New App β†’ From scratch.

  2. Pick a workspace and name it AdSentry.

  3. In the left sidebar, open Incoming Webhooks and toggle it On.

  4. Click Add New Webhook to Workspace, choose the channel (e.g. #adops-alerts), and Allow.

  5. Copy the webhook URL (it looks like https://hooks.slack.com/services/T.../B.../...).

  6. Set it in your environment:

    export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/T.../B.../..."
  7. Fire a test:

    node dist/index.js check --simulate

You'll get a red, header-topped Block Kit message with per-campaign fields (spend, CPA, baseline CPA, overrage %) and a <!here> mention.

πŸ’‘ Want to mention a specific person/group instead of everyone? Set SLACK_MENTION="<@U012ABCDEF>" (a user) or SLACK_MENTION="<!subteam^S012ABC>" (a user group).

Discord

  1. In your server: Server Settings β†’ Integrations β†’ Webhooks β†’ New Webhook.
  2. Choose the channel, Copy Webhook URL.
  3. export DISCORD_WEBHOOK_URL="https://discord.com/api/webhooks/..."
  4. Optionally set DISCORD_MENTION="@here" or a role mention like DISCORD_MENTION="<@&ROLE_ID>".

🧬 Anatomy of an alert

When a campaign trips a rule, the detector attaches the full reasoning, and the notifier renders it:

🚨 CRITICAL β€” CPA Spike Meta Ads β€’ sim_meta_runaway_002 [SIM] Meta β€” Advantage+ AUTO (RUNAWAY)

Spend Conversions Current CPA Baseline CPA (7d) CPA vs baseline
680.00 USD 1 680.00 USD 14.00 USD 4857.1%

πŸ” Current CPA 680.00 USD is 4857.1% of the 7-day baseline CPA 14.00 USD (threshold: 150%). The bidder is paying far more per conversion than it historically should.

Every alert shows its work β€” no black-box "something's wrong," just the exact numbers an on-call engineer needs to act in seconds.


πŸ—οΈ Architecture

src/
β”œβ”€β”€ types.ts      # Strict domain model: snapshots, baselines, thresholds, verdicts, payloads.
β”œβ”€β”€ fetcher.ts    # Meta + Google API clients, resilient HTTP (backoff/429), simulation mode.
β”œβ”€β”€ detector.ts   # The rule engine: CPA math, baseline comparison, severity, explanations.
β”œβ”€β”€ notifier.ts   # Slack Block Kit + Discord Embed builders, resilient webhook delivery.
└── index.ts      # commander CLI: `check` (one-shot) & `watch` (cron daemon).

Design principles:

  • Explainable by construction. Every AnomalyResult carries the math behind the verdict β€” and the reason it didn't fire when healthy, for auditability.
  • Fail loud, never silent. If all fetches die, we fall back to simulation and still emit a heartbeat rather than going dark.
  • One bad cycle never kills the daemon. Each scan is wrapped; unhandledRejection/uncaughtException are caught and logged.
  • No overlapping scans. A run-lock skips a tick if the previous one is still running.

🧰 Development

npm run typecheck   # tsc --noEmit (strict)
npm run build       # bundle to dist/ via tsup (ESM, sourcemaps, d.ts)
npm run dev         # watch-rebuild
npm run check       # build output: one-shot scan
npm run watch       # build output: resident daemon

Run as a service (systemd)

# /etc/systemd/system/adsentry.service
[Unit]
Description=AdSentry ad-spend anomaly watcher
After=network-online.target

[Service]
WorkingDirectory=/opt/adsentry
EnvironmentFile=/opt/adsentry/.env
ExecStart=/usr/bin/node /opt/adsentry/dist/index.js watch
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
sudo systemctl enable --now adsentry
journalctl -u adsentry -f

Run via Docker / cron

# Hourly one-shot via host cron (alerts on its own; exit code reflects anomalies)
0 * * * * cd /opt/adsentry && /usr/bin/node dist/index.js check >> /var/log/adsentry.log 2>&1

❓ FAQ

Does it modify my campaigns or pause spend? No. AdSentry is read-only and alert-only by design. It observes and notifies; humans decide. (Auto-remediation is intentionally out of scope β€” you don't want a bot pausing campaigns on a false positive.)

Why compare to a 7-day baseline instead of a fixed CPA target? Because "normal" drifts. Seasonality, promos, and audience shifts move your CPA legitimately. A trailing baseline adapts; a static number pages you at 2am for nothing.

What if a campaign is brand new with no history? The detector treats an unreliable baseline (too few conversions) as "skip the spike check" and says so in the verdict β€” but the zero-conversion burn rule still protects you from a new campaign that spends with nothing to show.

Will it survive a Meta/Google rate limit (429)? Yes. Both the fetcher and notifier honor Retry-After, back off exponentially with jitter, and retry up to MAX_RETRIES before giving up gracefully.


🀝 Contributing

Issues and PRs are welcome. Keep the compiler happy (npm run typecheck), keep alerts explainable, and keep the daemon crash-proof.


πŸ“„ License

MIT Β© AdSentry Contributors

MIT License

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

AdSentry β€” your AI bidder doesn't sleep. Now your guardrails don't either.

⭐ If this saved your budget, star the repo.

About

πŸ›‘οΈ SRE-grade real-time anomaly detection & alerting for ad spend (Meta Ads / Google Ads). Catches CPA spikes & runaway AI campaigns, alerts Slack & Discord before your budget burns.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors