Skip to content
View vishalhabib99's full-sized avatar
🎯
AI PM & builder — I ship evals & guardrails for AI agents, red-teamed in public
🎯
AI PM & builder — I ship evals & guardrails for AI agents, red-teamed in public
7 followers · 0 following

Achievements

Achievement: Pull Sharkx2Achievement: QuickdrawAchievement: YOLOAchievement: Pair Extraordinaire

Achievements

Achievement: Pull Sharkx2Achievement: QuickdrawAchievement: YOLOAchievement: Pair Extraordinaire

Block or report vishalhabib99

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
vishalhabib99/README.md

Hi, I'm Vishal 👋

Lead AI Product Manager & Builder — Agentic AI · leading the 0→1 Agentic AI Digital Advisor at Vanguard, now in limited client beta · ex-T-Mobile, eBay · Stanford GSB

10+ years launching 0→1 products and scaling platforms 1→100 across fintech, SaaS, enterprise platforms, marketplaces & e-commerce: $9B+ in revenue platforms, 29M+ MAU, $300M+ in cost savings. On the side I build the eval and trust tooling I wish every AI team had, and publish the failures with the passes.

Outside proof: 83 fixes merged in 79 outside repos, including at Microsoft, AWS, NVIDIA, Intel, IBM and AMD Research (details).

▶️ 88-second walkthrough (AI-narrated):

video-FINAL-2026-10-29.mp4

🩺 Flagship: mcp-doctor, evals & guardrails for AI agents and the MCP tools they call

mcp-doctor answers two questions: can an agent actually use your server, and can you trust what the agent does with it? Scan your server →

  • 🔍 Scan: open an issue with your repo URL, get a report that leads with what will break an agent, then quality and security (example), no install. Or run it in CI with the GitHub Action, or as a plugin for Claude Code, Codex or Cursor.
  • 🛡️ guard: an unguarded agent read a web page, then planted a .git hook and leaked a .env token; guard stopped both. One line puts an MCP server or a coding agent's built-in tools (Claude Code, Codex, Cursor) behind a live check for likely prompt-injection leaks and "successes" that didn't happen.
  • 🧪 fuzz and verdict: launches a server and checks it fails cleanly, --truth checks a "success" is a real answer, and verdict turns it all into SHIP, FIX-FIRST or BLOCK, blocking any finding that will break an agent even when every grade is an A.
  • 🤖 fix-upstream: an agent that turns a finding into a tested PR I approve first. Replayed on 15 fixes maintainers had already merged, it made the same edits; its first real PR merged at AMD's intellikit in 41 minutes (write-up).
  • 📊 Leaderboard and census: 24 public servers checked; 3,923 servers and 147,646 tools scanned (write-up).

Where it stands: jupyter-mcp-server became the first outside CI user on Oct 16, after its maintainer asked for the Action. It finds 15,755 of 15,977 tools across 805 real servers, every miss published; by scanning real servers, I found and fixed 143 bugs in mcp-doctor itself (postmortem).

Animated mcp-doctor demo in four steps: a server grades an A but has a bug that will break an agent, the release verdict BLOCKs it, --fix fixes it and the verdict becomes SHIP, then guard catches a live call that reported success without writing the file

✅ Proof from outside

None of these maintainers work with me.

"Thank you for the exact tools/list results and for explaining the effect on clients that use annotations for approval." (codebase-memory-mcp maintainer)

🧱 More repos, by vertical

The common thread: AI output is worth shipping only when it can be checked against a source of truth: an IRS rule, a tool's schema, an API contract.

🏦 Fintech

  • 🛡️ fin-assist: catches stale and wrong money figures in an AI agent's answer before a customer sees it, checking each against 102 dated, sourced IRS, SSA and Medicare figures and flags advice and promissory language, citing IRS or FINRA.
    Blind-tested on eight agent-written sets, every miss published; a fact-checking judge cut wrong facts sent from 5 of 15 to 0 of 25. Its registry caught a real tax-bracket bug in ignidash, an open-source retirement planner: the maintainer merged my fix. pip install fin-assist · paste an AI answer to check it · model risk pack

🛒 Marketplaces & E-commerce

  • 🏷️ listing-claim-check: checks an AI-written listing against the seller's own item specifics and decides PUBLISH or REVIEW. Try it →
    v0.1 failed its blind run, v0.2 passed a fresh one, then a red team got all 22 attacks through, published as-is.
  • 🧩 eBay case study: the platform behind $300M+ in savings, and why clear contracts matter for agents.

🏢 SaaS & enterprise platforms

  • 🔗 agent-handoff-check: checks every agent-to-agent handoff against what the customer authorized and decides ACT, ESCALATE or BLOCK. Try it →
    A red team got 3 unauthorized calls through; after a redesign, a fresh blind run let 0 of 18 through and blocked 0 of 14 legitimate calls.
  • 📡 T-Mobile case study: four architecture decisions behind the agentic platform (25M users, 60% containment) and what I'd do differently.

🧭 For AI product managers

🗺️ Where my work sits in the AI stack: shipped at work vs. built in the open
Layer Shipped at work Built in the open
Apps & human-in-the-loop 🏦 Vanguard Digital Advisor experience · 🏢 T-Mobile agentic support platform 🏦 Contribution room calculator · 🏦 Human review queue · 🛒 Listing checker demo
Agents & orchestration 🏦 Agentic AI Digital Advisor (0→1) · 🏢 Autonomous enterprise agent platform GuardedSession: ACT, ESCALATE or BLOCK on each live tool call · 🏢 agent-handoff-check: authority can only narrow across handoffs
Tools & APIs (MCP) 🛒 eBay API standardization across hundreds of teams mcp-doctor and its fuzz · 🏦 check_answer and contribution_room MCP tools · 🛒 check_listing MCP tool
Evals & quality 🏦 Model evals for correctness, groundedness, safety, latency · 🏢 IntentCX evaluation framework 🏦 Blind, pre-registered evals · 🛒 Blind pass, then broken by a red team · 🏢 RAG prototype on blind tickets: accuracy fell from 60% to 38% · ⚛️ Quantum circuit grader: right on all 90 real answers, yet 4 red teams got 27 wrong circuits passed (write-up) · /eval-plan
Guardrails, governance & risk 🏦 FINRA/SEC-compliant responsible AI design · 🏢 Governance aligned to NIST AI RMF 🏦 Model risk pack (SR 26-2 + NIST AI 600-1) · 🏦 Prompt-injection red team, before and after the fix · mcp-doctor verdict release gate · guard for coding agents · 🏢 Red-teamed handoff checks
Cost & pricing 🏢 Accuracy, cost and latency tuned per interaction type 🏢 Which model, and how to price it
Product decisions 0→1 strategy, launch gates, adoption and containment metrics PRD: Agent Outcome Trust Score · /build-or-not · Agent Readiness Scorecard · What I decided not to build

💼 Career, by vertical

Vanguard · T-Mobile · eBay · earlier roles
  • 🏦 Fintech: Vanguard. Product strategy for Digital Advisor and Personal Advisor ($6B+ LOB, 4M+ MAU) at the world's second-largest asset manager (~$12T AUM). Architecting the Agentic AI Digital Advisor from 0→1, now in limited client beta (7+ use cases, launch gates met): LLM orchestration, agent workflows, model evals and FINRA/SEC-compliant responsible AI. Also launched T-Mobile Money and, earlier, digital lending modernization at Axis Bank.
  • 🏢 SaaS & enterprise platforms: T-Mobile. Product lead for T-Life, the flagship app ($3B+ LOB, 25M+ MAU); mentored 3 PMs and led a 40+ person org. Launched one of the first autonomous enterprise Agentic AI platforms in US telecom: 75% adoption, 60% containment, 80% CSAT, 30% fewer support calls. Built the IntentCX AI governance and eval framework (NIST AI RMF), adopted by 3 more teams. AI personalization: +27% engagement, +15% conversion.
  • 🛒 Marketplaces & E-commerce: eBay. Drove $300M+ in savings and +35% adoption through platform modernization and API standardization across hundreds of teams; 40% faster deploys, 99.9% availability on seller APIs.
  • 🩺 Earlier, healthcare: Premera Blue Cross. Billing and payment redesign that cut task completion time 25%.
Recognition
  • Vanguard AI Summit (2026): built and presented the AI Agents Evaluations & Guardrails Framework, now adopted by 6 Vanguard orgs.
  • Patent filed: sole inventor on a U.S. provisional patent application (No. 63/980,243, May 2026) for agentic AI orchestration and governance.
  • Top Product Leader, T-Mobile (2025), for building and launching the enterprise agentic AI platform.
  • Keynote speaker, T-Mobile Technology Innovation Summit (2025): AI strategy keynote to 1,200+ attendees.
Education & certifications
  • Stanford Graduate School of Business: Executive Program, Harnessing AI for Breakthrough Innovation and Strategic Impact (2026)
  • MBA, Business Strategy and Marketing, Indiana University of Pennsylvania · MS, Information Technology Management, Campbellsville University
  • Certifications: PMP · SAFe POPM · AWS Solutions Architect Associate · CSPO · PMI-PBA · Google AI Essentials
What this portfolio doesn't cover
  • Semantic hallucination in general needs a judge model; only fin-assist uses one, for IRS and FINRA fact checks.
  • Live agent red-teaming: guard catches leaks live, but nothing here attacks an agent to find new holes.

📫 Reach me

Email · LinkedIn · Website · dev.to · Book 1:1 time

Pinned Loading

  1. mcp-doctor mcp-doctor Public

    Evals & guardrails for AI agents and the MCP tools they call: lint + auto-fix, fuzz, SHIP/FIX-FIRST/BLOCK verdict, live guard. Runs in CI, Claude Code, Codex, Cursor.

    Python 3

  2. fin-assist fin-assist Public

    Catches stale and wrong money figures in AI agents' answers before customers see them: 102 IRS, SSA and Medicare figures by year, each with its source. pip install fin-assist, MCP server, browser c…

    Python 1

  3. listing-claim-check listing-claim-check Public

    Checks AI-written marketplace listings against the seller's item specifics before they're published: PUBLISH or REVIEW, with the exact words behind every unbacked claim. Deterministic, blind-evalua…

    Python 1

  4. agent-handoff-check agent-handoff-check Public

    Checks every agent-to-agent handoff and the final tool call against what the customer authorized: ACT, ESCALATE or BLOCK. Authority can only narrow. Deterministic, blind-evaluated, red-teamed.

    Python 1

  5. ai-pm-skills ai-pm-skills Public

    Claude Code skills for AI product managers: /build-or-not, /eval-plan, /agent-trust-review. Each tested with evals, failures included.

    Shell 1

  6. circuit-claim-check circuit-claim-check Public

    Checks whether a quantum circuit an AI agent wrote is the circuit it was asked for: exact Qiskit grading, no model in the loop. Includes four red-team rounds and every finding.

    Python 1