ZeroNoise Logo zeronoise
Post
Optimizely compromise: product teams own failures in vendor code too
•
4 min read
• 110 docs
The Optimizely script injection and a run of outages raise questions about vendor risk and failure-state design. Also: simulation as a research tool, the PRD as a decision record, growth loops, and pricing pages written for AI agents.

A compromised A/B testing tool, and what product teams should take from it

Mind the Product's weekly review leads with Optimizely, the experimentation platform many product and growth teams use. Report URI found malicious JavaScript served to visitors of nearly 200 companies, including Zoom, the BBC, CNN and Salesforce. An attacker had put a loader into the field where customers can add their own custom JavaScript. Of 1,611 scripts checked, 188 were infected, across 178 organizations . The payload swapped crypto wallet addresses, skimmed card numbers, and took cookies and screenshots . Optimizely blamed a flaw in the signup flow for its web experimentation product. It said fewer than 5% of customers were affected and that "this was not a credentials breach" . Allowing Optimizely's domain in a content security policy protected nobody, because the bad code came from that domain .

The host is careful to say he can't prove AI caused any of the week's failures . His argument is about volume. He cites Veracode: AI writes roughly half of committed code at adopting companies, and about 44% of AI-generated tasks introduced a known vulnerability . He also links this to flatter orgs. Canva-style cuts to layers may also remove the people who caught each other's mistakes .

His recommendations:

  • Own vendor failures. Customers blame your brand, not the vendor . Write a one-page plan covering who detects a failure, who decides what customers are told, and who writes and approves the message. Then run it as a drill .
  • Design the failure state. Pick one critical vendor and write down what users see and can still do during a 24-hour outage. If the answer is "nothing," you've made a product decision by accident .
  • Gate prototypes. "The AI wrote it and the tests pass" is not a definition of done. Someone has to understand the code before it ships .

Hiten Shah describes the same review problem from the builder's side. After a day with 3,120 commits from agents, his question became "how am I supposed to review all this work?" . He sums it up: software is getting easier to produce faster than people are getting better at deciding what should exist .

Simulation as market research

Scott Belsky writes about startups such as Simile that build simulated populations to test slogans, pricing and other decisions. He says many marketing and price-testing uses aimed at specific demographics are proving more than 90% accurate when projections are compared with outcomes, though the field is early . Product uses he names include whether current customers would pay for an adjacent product, and how a price increase would affect brand sentiment . He expects simulations to replace focus groups and some market research. His caveat: simulations reflect how people think and feel today, so they may miss the edge preferences that later become the center . Shah adds a practice: run the scenarios, make the decision, then study what the simulation got wrong, because "the best ideas might be hiding in the misses" .

The PRD as a decision record

Aakash Gupta's updated template assumes you go from an idea to five prototypes and write the PRD for the one that survives. The PRD becomes "a decision record more than a permission document" . What's new for AI features:

  • a behavior contract of 15–25 example responses, ranging from bad to good to excellent
  • graduation metrics plus a golden set of offline evals
  • the rollout's exposure, duration and size, plus kill-switch conditions and an owner
  • a link to results at the top of the document

In a separate note, he says AI-built apps still need a growth loop. LinkedIn's contact imports, Substack readers who become writers, and Canva designs that act as ads are product decisions made before any code is written .

Smaller signals

  • Agent-readable pricing. A product marketer says developers now ask an agent before opening docs or pricing. In their check, 15 of 16 developer tools served a markdown pricing page, compared with 1 of 12 SaaS products. They ask who should own how agents see the product, and whether anyone tests what ChatGPT or Claude says about their pricing .
  • A discovery lesson. Vedion, an execution-risk diagnostic tool, reached three paid pilots and then shut down. Its founder says leaders already knew where execution was breaking. What they lacked was internal ownership and follow-through, which nobody would pay an outside tool to provide .
  • Dev-tool niches. Commenters say generic AI code reviewers are crowded. Narrow niches like messy industry data or compliance are more open, because the users won't just build the tool themselves . One test: ask 10 engineers what they still do by hand. If three name the same pain and no tool owns it, build that .
  • AI PM jobs. Gupta lists 34 open PM roles at model companies, led by Tencent (7), Google DeepMind (6) and Alibaba (5). Posted base pay reaches $460K at Anthropic and $445K at OpenAI .
Optimizely compromise: product teams own failures in vendor code too
Mind the Product
  • Optimizely’s web experimentation product was abused through a signup-flow flaw that let an attacker impersonate users on customer accounts and inject JavaScript; Report URI found 188 infected scripts among 1,611 checked, across 178 organizations. The payloads included card skimming, cookie and screenshot theft, and cryptocurrency-address swapping; because the code came from Optimizely’s trusted domain, allowing that domain in a site’s content security policy did not protect it.
  • Plan incident ownership before a failure: decide who detects it, who has authority to act, and who drafts, approves, and sends customer communications; write and rehearse a one-page response plan. Customers may hold the product brand responsible even when a vendor caused the problem. For critical vendors, specify what users will see and still be able to do during a 24-hour outage; having no fallback is itself a product decision.
  • The episode says AI has not been shown to cause the outages discussed. For teams using AI coding tools, set a clear prototype-to-production gate and require someone to understand the code before shipping; AI authorship and passing tests alone are not a definition of done.
What went wrong at Microsoft, Workday and the FAA?
Product Management
  • For discovery, one PM described scanning competitors’ public social channels, roadmaps, announcements and support materials, alongside win/loss transcripts, demo sites and customer-published workflows; their user-problem research includes customer conversations, purchasing-trend whitepapers, industry events and support-ticket trends.
  • One PM found that supposed beta customers were only account names, with no qualified use case, working owner, success criteria or testing plan, and had to build the qualification model, beta structure, roadmap and positioning.
  • A senior PM’s launch-week example shows the coordination load: after a partner’s last-minute requirements change, leadership chose to make the change; the PM then coordinated launch preparation, updated documents, escalated customer-email approvals, handled customer and account-team requests, and routed a beta issue to engineering.
Sort of a lame question as any book can answer this. But it's fun because my answer has changed so much over time. Research competitors: … * Prepare to demo a product to a major enterprise customer. * Receive access to the demo environment the day before. * Realize nobody has… We have a launch on Monday so all I did Thursday was work with solution architects to write a blog which got reviewed by like 5 people wi…
Aakash Gupta
  • Aakash Gupta lists 34 open AI PM roles at model companies and says all work on the models themselves; the largest listed groups are Tencent (7), Google DeepMind (6), and Alibaba (5). The note reports posted base salaries up to $460K at Anthropic and $445K at OpenAI, with stock on top.
  • For AI products, Gupta recommends moving from an idea to five prototypes, then writing the PRD for the selected prototype as a decision record. Include a behavior contract with 15–25 examples spanning good, bad, and excellent responses, plus success metrics and a golden set of offline evaluations. Record rollout exposure, duration and size, kill-switch conditions and owner; keep the PRD evolving by linking results at the top.
  • AI makes building easier but does not solve user acquisition: a funnel needs continued traffic, while a product loop can generate the next users. Examples include LinkedIn inviting imported contacts, Substack readers becoming writers, and Canva designs acting as ads; the PM task is to identify the loop before building.
  • Useful design heuristics include clear visual hierarchy, consistent spacing and alignment, restrained type and color, and body-text contrast of at least 4.5:1. Make actions look clickable, show progress and clear outcomes, design empty/loading/error/partial states, and use smart defaults. Keep primary actions thumb-reachable and motion under 300 ms.
These are the 34 open AI PM jobs at the top AI model companies right now. Every one of them works on the model itself. Here's where to ap… AI product managers, save this PRD template. (Your old one is out of date.) When I started in product management back in 2008, the PRD wa… AI can build your entire product now. It still can’t get you a single user. Those were never the same job. Someone on r/ClaudeAI just lai… 12 golden design concepts every PM should know 1. Hierarchy - Make the one thing that matters the biggest, boldest thing on the screen, s…
Hiten Shah
  • Hiten Shah predicts software will make simulation a callable primitive for agents: rehearse scenarios before decisions, compare predictions with outcomes, and study misses for new ideas.
  • For product teams, simulated populations could test adjacent-product demand, pricing, and marketing copy; Scott Belsky says some demographic-specific marketing and price tests are already proving 90%+ accurate, and predicts simulations may replace focus groups and some market research. Belsky cautions that simulations reflect past behavior and how people feel today, raising the question of whether human intuition is still needed to spot preferences that may later become mainstream.
Software is going to rehearse decisions before making them. Simulation becomes a primitive every agent can call. Run the scenarios. Make … Decision By Simulation: What Can Go Right & Wrong
scott belsky

Population-simulation startups combine models with broad structured and proprietary data to test decisions such as whether customers would pay for an adjacent product, how price increases could affect brand sentiment, and how marketing copy or price points may perform. Belsky says some demographic-specific marketing and pricing tests are 90%+ accurate against outcomes, while describing the field as early-stage. Belsky flags a strategic limitation for product research: simulations reflect past behavior and how people think and feel today, so human intuition and creativity may be needed to spot emerging preferences before they become mainstream—and to help brands shape taste rather than simply follow it.

Decision By Simulation: What Can Go Right & Wrong
Hiten Shah

After a day with 3,120 commits from agents, Hiten Shah said the challenge quickly shifts from producing work to reviewing it; he wants to see what agents built, make the important calls, and let them continue.

We’ve had a day with 3,120 commits from agents. It’s ridiculous how quickly “look at all this work” turns into “how am I supposed to revi…
The community for ventures designed to scale rapidly | Read our rules before posting ❤️

A practical product-governance takeaway for prospective cofounders is to define decision rights by domain before committing: one suggested split assigns sales and pricing to one partner and product decisions to the other, with hiring and spending decided jointly; also settle how deadlocks and buyouts will be handled.

Is it easier to partner with someone who has the skills that you lack, or learn them yourself? What if said partner leaves? (I will not promote) The skills split is fine, plenty of good pairs are one builder and one seller. What I'd push on is him wanting final say on everything be… the skills gap is solvable, the decision rights thing is the part i'd worry about. "one person has final say" is fine in principle, but i…
Product Management - The place for all things product

Feedback on a product-management consultant’s portfolio site emphasized demonstrating distinctive work: a reviewer said it lacked clarity, credibility, and proof, and urged showing what the consultant had done that was unique; another reviewer called it “AI slop.”

I built a portfolio site to explain how I approach product problems — looking for honest feedback It reads as AI slop. No proof at all. And the link shared went to a not here page. No clarity. No credibility. Nothing you say you do is … I don’t want to sound rude, but I am a product manager who spent countless hours building prototypes using AI. Though you might have spen…
Hiten Shah

Hiten Shah argues that software is getting easier to produce faster than people are improving at deciding what should exist, underscoring product judgment as a growing constraint on building software .

Software is getting easier to produce faster than people are getting better at deciding what should exist.
The community for ventures designed to scale rapidly | Read our rules before posting ❤️

For developer-tool product discovery, commenters distinguish crowded generic products such as AI code reviewers from niche workflows involving an industry’s messy data or compliance work; one recommends interviewing 10 engineers in a niche, looking for the same manual task recurring among several, and checking that no tool already owns it and users would pay rather than build it themselves.

it's crowded at the general level, like yet another ai code reviewer. it's still pretty open in boring niches where the users aren't devs… crowded idea lists are a trap. the durable wedge is usually a nasty workflow inside one stack that incumbents wont touch, not another gen…
Product Management - The place for all things product

A PM with several years of service-industry experience was looking for structured product-lifecycle-management learning through a company-paid Coursera license . One commenter, drawing on taking at least five Coursera courses, said they added little practical value and recommended Aakash Gupta’s videos and free ccforpms courses for upskilling .

Corusera product management courses It’s the shittiest education site built in the world. I took at least 5 courses and they add no value to practical skills. If you really …
The community for ventures designed to scale rapidly | Read our rules before posting ❤️

Vedion, an execution-risk diagnostic for PE-backed companies and scaling startups, reached three paid pilots before its founder wound it down. Pilot and operator conversations suggested that leadership teams already knew where execution was breaking; the unmet need was internal ownership, cadence, and follow-through, and customers were not going to pay an outside tool to provide that. This is a product-discovery caveat: paid pilots for a diagnostic did not establish demand for an external tool to solve the execution problem.

Ex-founder, 6+ yrs in consulting and 0-to-1 building, looking for a remote Founder’s Office or a Chief of Staff role [I will not promote]
ProductManagementJobs

An ML engineer with 8+ years in personalization and recommendations, who says about half their work is product-related, reports getting no responses to product-management applications despite emphasizing business impact and domain fit; they suspect not having held a formal PM role may be a barrier. This is an individual job-search experience, not evidence of a broader hiring trend.

Landing a Product Job as an MLE
ProductManagementJobs

A Product Manager opening is advertised for candidates with 3+ years’ experience; AdTech experience is preferred. The post links to the original listing in r/Indiajobs.

Product Manager Opening (3+ years and AdTech exp preferred)
Product Marketing

A product-marketing post says developers increasingly ask AI agents before opening product documentation or pricing pages. In the author’s check of 28 pricing pages, 15 of 16 developer tools and 1 of 12 SaaS products served a Markdown version; the post raises whether teams should assign ownership for agent-facing product information and test how ChatGPT or Claude describes pricing .

Are you writing for AI agents yet, or only for people?