We can't find the internet
Attempting to reconnect
Something went wrong!
Hang in there while we get back on track
Building is cheap; verifying and keeping a lead are now the hard parts
•
4 min read
•
186 docs
Hiten Shah's essay on what happens when models catch up, Bolt's view of the PM job when building is nearly free, and the verification problem both point to. Also: a weak PM hiring-replacement rate and AI feedback-triage practices.
When the models catch up
Hiten Shah's new essay uses Gamma to separate finding a customer problem from being able to solve it. Gamma interviewed 100 people before building. They named the same frustrations: the blank page, hours spent formatting, and worry that a weak deck would make them look less capable . The first product made arranging slides easier, but users still had to write most of the content. Co-founder Jon Noronha estimates, looking back, that only about 5–10% of signups reached value . After a three-month rebuild around generative AI, users could describe an idea and react to a GPT-4 draft . Shah says the result can't be credited to that change alone, because onboarding and infrastructure changed at the same time .
His main lessons for PMs:
- A demo isn't a product. In his illustration, five independent steps that each succeed 80% of the time finish the whole job only about 33% of the time. At 95% per step, it's about 77% . The product has to make the model dependable through context, tools, verification and recovery, at a price customers will pay .
- Scaffolding expires. Layout restrictions that made 2023 models usable were later holding stronger models back. What has to stay dependable is the customer's commitments: saved work, permissions, editability .
- Before the models are ready, ship work customers can finish today. Watch where they get stuck and whether they come back, and log failures at the level of complete tasks so they become reusable evals. Moats such as customer context, integrations and distribution are earned through use . Shah points to KISSmetrics, which was early in its market until competitors closed the gap: "Being early gives you time" .
He also posted a prompt to run after a coding agent builds a feature. It asks the agent to list up to three consequential user-facing choices you didn't specify, with evidence, without changing anything. Examples include whether users can skip signup or what happens when a customer cancels .
The PM job when building is nearly free
Bolt's Eric Simons argues that the classic product process was a "rationing protocol" for expensive software. Now PMs can skip low-fidelity prototypes and build the real thing . In Bolt's survey of PMs who use Bolt, 38% now bring engineering in earlier with a real prototype, and 30% bring engineering in only after validating the idea . In his view, the lasting advantage is customer context: ongoing conversations and pain points, not "a golden idea" .
The bottleneck moves to verification. Blacksmith's founder says writing 100x the code doesn't help if you can't validate it, and the company is building autonomous QA that collects screenshots and video as evidence . A principal engineer on Reddit says he can produce 100 PRs a week, but finding the roughly 10% that are bad "is such a slog" . A caution for anyone building in this space: Synathic, an agent-write verifier, got no trials from 8 design partners because people patch the problem themselves cheaply. Its builder's takeaway is that adoption cost decides everything and that detection latency matters more than failure type .
PM careers: fewer backfills, more negotiation
Aakash Gupta reports that in 2025, companies hired 0.92 outside PMs for every PM who left, meaning roles were closed rather than refilled . His negotiation playbook:
- Set walk-away, target and anchor numbers before the first call .
- Let the employer name a number first .
- When base pay hits the top of the band, trade for sign-on, equity or an earlier review .
All five 2026 cohort negotiations he wrote up got more money, and none lost the offer . He also shares a heuristic from Mark Pincus: a change is "better" only if 10 out of 10 users would pick it; otherwise call it "new" .
Practices worth stealing
- AI feedback triage with guardrails. One PM runs a weekly Claude automation that themes feedback, sends uncertain items to human review, and never lets the model create or change themes . Another commenter recommends labeling 100 items by hand and using them as an eval to compare models over time .
- Frame the outcome first. Strategyzer's new Problem Framing Map rests on the idea that "the problem is when you can't get to where you want to go." In their example, contractor interviews showed nobody owned the end-to-end system, which led to very different fixes .
- A daily pre-mortem. Reed Hastings spent 10 minutes a day assuming Netflix had crashed and asking what he did to cause it. He calls it "a quiet little reservoir of darkness" .
Building is cheap; verifying and keeping a lead are now the hard parts
Hiten Shah
- Gamma’s August 2022 Product Hunt launch won Product of the Day, Week, and Month, but co-founder Jon Noronha later estimated that only about 5–10% of signups reached value. Interviews with 100 people exposed blank-page and formatting frustrations, while the original editor still left customers to create most content and structure the story. After a three-month rebuild around generative AI, users could describe an idea and receive a first draft; the relaunch ranked seventh, but the comparison is not controlled because the product, onboarding, and infrastructure also changed.
- AI product reliability depends on the whole customer task, not a compelling demo: in an illustrative example, five independent steps each succeeding 80% of the time yield about 33% end-to-end success, versus about 77% at 95%; the article cautions this is not a real-product reliability prediction. Product teams must make model capability dependable through context, tools, verification, recovery, and viable cost.
- Product scaffolding must evolve with models: Cognition changed its prompting after Sonnet 4.5 exposed assumptions in its agent, while Gamma found that restrictions useful for weaker models were holding stronger ones back. Keep customer commitments such as saved work, permissions, and editability dependable even as temporary technical constraints change.
- Before models are ready, ship useful work customers can complete; observe where they get stuck, which results they accept, repair effort, and whether they return. Track failures across complete tasks—including models, tools, data, tests, and human intervention—to build reusable evaluations. Shah says KISSmetrics was three years ahead but his shifting priorities let that lead disappear; model access likewise does not secure a lasting advantage, so customer context, integrations, distribution, and workflow knowledge must be earned through use.
Hiten Shah
- Gamma interviewed 100 people before building; they described blank-page starts, time lost formatting slides, and fear that a poor presentation would make them seem less capable. Its first browser-native editor made arranging and sharing easier, but still left users to create most of the material and structure the story . Noronha later estimated that only about 5–10% of initial signups reached value; the GPT-4 relaunch let users describe an idea and react to a generated draft, though the article cautions that its outcome cannot be attributed to one change because the product, onboarding, and infrastructure also changed .
- Treat a compelling AI demo as distinct from a dependable product: a customer task can require several successful steps, and the article’s illustrative example puts five independent steps with 80% success each at about 33% end-to-end success. The product must make model capability dependable through context, tools, verification, recovery, and affordable delivery; the calculation is illustrative, not a prediction for a real product .
- Revisit model-dependent engineering as capabilities change: Gamma found restrictions that helped earlier models generate readable presentations were holding stronger models back. Separate temporary scaffolding from durable customer commitments such as saved work, permissions, and editable presentations .
- When an automated first output becomes commonplace, look for the next unresolved customer need: Gamma shifted toward brand-specific design, editing, and story development, while Lovable broadened from generating applications toward deploying and operating them .
- If the full promised outcome depends on future models, begin with work customers can finish today; observe where they get stuck, what they accept, how much repair is needed, and whether they return. Track failures across complete tasks—including models, tools, data, tests, and human intervention—as reusable evaluations, and earn differentiation through customer context, integrations, distribution, workflow knowledge, and verification . Reliability, cost, and recovery requirements should also reflect the task’s stakes .
Scott Belsky
Profile
- Doritos responded to a declining brand by giving fans control over making and submitting Super Bowl ads: the first contest drew more than 1,000 entries, with five finalists receiving $10,000 each; over the contest’s 10 years, it generated more than 30,000 fan-made ads. As a crowdsourcing pattern for product teams, this shows how user contributions can create a large pool of ideas, but the hit rate was low: the speaker says 98% of submissions were not usable or airable. The payoff was uneven—the retrospective characterizes it as a great opportunity for about 10 people and a free test for the other 30,000—making participant incentives and fairness important design considerations.
Lenny Rachitsky
- For B2B SaaS, Wispr Flow’s experience suggests trying paid UGC once the product has PMF and organic pull: paid can quickly test whether messaging resonates and cold traffic converts, while product quality remains the underlying growth driver.
- Wispr Flow pays creators a percentage of the ad spend behind their ads. To keep new creators engaged while they improve, it uses a “smart retainer”—for example, a 25-video quota guarantees $1,500, with commissions offsetting the guarantee; strong creators can later move to commission alone.
- Recruit creators who speak to distinct ICP niches and encourage them to experiment rather than copy top-spend ads: Hector warns that similar ads fatigue quickly, and Wispr’s creators target audiences ranging from lawyers and engineers to moms and sales reps.
Aakash Gupta
- In 2025, companies hired 0.92 external PMs for every PM who left; the post characterizes this as roles being closed rather than refilled.
- For PM offers, set walk-away, target, and anchor numbers and research the pay band first; ask the employer’s range before naming your number, and request the offer in writing rather than accepting on the call. If base pay is capped, consider trading for sign-on, equity, an earlier review, or non-salary terms; sign-on is described as easiest to move, stock as more movable at senior levels, and level as mostly set in interviews.
- Aakash reports that all five 2026 cohort negotiations he wrote up resulted in more, with no offers withdrawn; two people were unemployed when they negotiated.
- Mark Pincus’s product guidance: call a change “better” only if all 10 users would choose it; otherwise treat it as new, and use frameworks as lenses rather than dogma. He also advises pursuing less new work early in a platform shift; his 80–90% download-step drop-off figure comes from Zynga Poker and is explicitly not a benchmark.
- For experiments, set the click-through success threshold before seeing results. Pocket FM CEO Rohan Nayak cited a move from 2% to 2.25% CTR alongside about 30% lower customer acquisition cost.
Hiten Shah
Hiten Shah called the linked playbook “epic” and “must read.” Product teams can apply three ideas from it:
- Define the customer before positioning: the same product may need different promises for different customer segments, and messaging should lead with the outcome rather than features or deliverables.
- Ground discovery and messaging in customer evidence: map customers’ current realities, frustrations, fears, and goals; talk to customers and use their own words; sales-call transcripts can surface purchase reasons, desired outcomes, pain points, and objections.
- When refining an offer, identify which of speed, ease, or risk is weakest and improve that dimension; copy can communicate an offer’s value but cannot create value that is not there.
Masters of Scale
- Netflix’s 2011 Quickster split came too early: separating DVD and streaming created an effective price increase, and Hastings said the company should have grandfathered existing customers; its focus on moving to streaming pushed it to drop DVD too abruptly.
- Hastings described a daily 10-minute practice of assuming Netflix had failed and asking what caused it, to counter the blind spots that can come with leaders’ optimism and surface negative scenarios.
- Netflix initially underestimated the value of original content: an earlier attempt failed because the company lacked distribution scale, and Hastings thought making content would lose money. He later let Ted Sarandos take home-run bets; Orange Is the New Black and House of Cards helped establish Netflix in the content business.
- Hastings said AI tutoring is currently not very useful, but expects it to improve. His product vision distinguishes answer delivery from learning: an AI tutor should quiz and question learners, make learning playful and age-appropriate, and use suitable training data and motivational design; he expects generalized tutors within two to four years.
Mind the Product
- Bolt’s founder argues that AI coding tools let PMs move beyond specs and tickets to build usable, high-fidelity prototypes for customer feedback and stakeholder alignment. Low-fidelity prototypes remain an option, but direct builds can be faster and avoid some of the interpretation loss of specs and wireframes.
- In a survey of PMs using Bolt, 38% reportedly brought engineering in earlier with a real prototype, while 30% involved engineering after validating an idea; the speaker says some prototypes were made before design or engineering work.
- The speaker says a PM’s durable advantage is customer context: ongoing conversations and understanding of customer pain points, technology, design, and adoption inform better ideas and execution. AI makes it easier to build and test solutions, but does not replace discovery and validation.
Y Combinator
- Blacksmith argues agentic coding is shifting the bottleneck from writing software to validating it and getting confidence that it works; it is developing autonomous QA that collects evidence such as screenshots and video snippets so developers can merge changes faster.
- Blacksmith differentiates its CI product through infrastructure tailored to CI workloads and by placing cached artifacts close to compute.
- Blacksmith reports that LangChain’s wait to acquire CI compute fell from 15–20 minutes on GCP to 10–15 seconds on Blacksmith. The company says it keeps the product surface area small and avoids bespoke customer requirements, scaling that surface area with the team; it credits this focus with supporting many customers and machines with a small team.
Lenny Rachitsky
Wispr Flow adapted e-commerce paid-UGC for B2B SaaS: creators make ads and receive a cut of ad spend, with its top creators earning $100K+ annually. The post associates this program with 30x+ revenue growth over the past year and a recent $2B valuation. A related clip presents Hector Serrano’s case that creators with smaller followings can be better investments for companies than big influencers.
Product Management
- Normalize feedback from different channels into per-item records (for example, ID, source, summary, themes/tags, and sender), then generate weekly views by theme and customer. A lightweight prototype can start with inspectable files or JSON and move to a database for more robust multiuser use.
- Keep the taxonomy human-owned and route uncertain classifications to human review. Treat AI theme analysis as directional, not a single source of truth: use it alongside customer outreach and discovery, and check classification quality against a manually labeled sample (for example, 100 inputs), tracking performance and comparing models over time.
- One commenter argues that scheduled jobs and AI make in-house feedback analysis feasible for someone with 3–5 years of analytics experience, raising the bar for SaaS tools to be cheaper and more feature-rich to justify buying.
Strategyzer
- Strategyzer launched its Problem Framing Playbook on its platform. It provides step-by-step guidance, short just-in-time videos and collaborative activities; teams frame a challenge, test assumptions through structured sailboat interviews and other experiments, then refine the map using evidence, with AI workflows to analyze findings.
- The map frames a challenge around stakeholders’ jobs, desired outcomes, current reality, blockers and enablers, and metrics. Its central guidance is to define what success looks like before selecting solutions, rather than relying on an abstract problem statement.
- In the webinar’s healthcare-website example, interviews with contractors challenged the initial assumptions: coordination was outside their scope, and each focused on completing their own piece. Reframing the issue as a lack of end-to-end ownership led to different solutions, including coordinating the contractors’ overall workflow and putting a contractor in charge.
The community for ventures designed to scale rapidly | Read our rules before posting ❤️
Startup growth should be evaluated against customer demand and business outcomes rather than headcount alone: a commenter warns that headcount increases burn without increasing valuation, while another says hiring should be tied to actual customer demand and revenue support. The thread’s suggested checks include ARR and burn rate, runway, and whether customers stay.
Product Marketing
A B2C product marketer with seven years in the field argues that AI could squeeze PMM’s intermediary role: product and leadership teams may generate messaging themselves, while channel owners can ideate and test quickly enough to reduce the need for stakeholder feedback. The commenter is trying to move into growth and views channel ownership as more durable.
The community for ventures designed to scale rapidly | Read our rules before posting ❤️
- Synathic, an open-source Python decorator that checks agent database writes with SQL, had no trials among 8 design partners; about 19 of 20 Reddit comments said people already handled the problem with a few lines of code or a cron job, although users also described green checks masking empty or stale results for days. The builder’s product lesson was that cheap workarounds make adoption cost decisive: an independent read is a stronger check but requires more setup, while detection latency (days versus minutes) is a more useful metric than failure type.
- For customer discovery, focus on users’ actual struggles and workflows rather than asking what they would pay for; stated pain can be noisy, overstated, or not yet recognized, so investigate how they currently solve the problem.
- A principal engineer building agentic development pipelines described verification as a bottleneck: they can generate 100 PRs a week, estimate about 10% may be bad, and find identifying them a slog.
Hiten Shah
Product teams should audit coding agents’ implicit product decisions after feature implementation: ask the agent to compare the original requests with the implementation, identify up to three consequential unspecified user-facing choices, show evidence, flag missing context, and make no changes. Use the same project so the agent can access the original requests. These choices can include whether users can skip signup or what happens when a customer cancels—decisions users experience even if the agent made them.
The community for ventures designed to scale rapidly | Read our rules before posting ❤️
- For a localized Rocket Money-like product, validate the job before coding: manually review a handful of people’s bank statements for forgotten subscriptions, and look for repeat requests or existing payment for a clunky workaround as evidence of demand.
- Check country-specific bank/card-data access and banking or regulatory constraints before assuming the U.S. product can be localized; commenters identify these as potential obstacles to the underlying service.
- A broader discovery loop: start with a problem and desired customer transformation, generate possible solutions, then test risky assumptions against data, customer insights, and experiments—quickly, over days rather than weeks.
a16z
- Tutor Intelligence treats robotics as a full-stack product problem: progress depends on trade-offs across models, hardware, software, supply chain, customers, and product, with successful business units serving as proof that the platform works.
- Tutor favors a general-capability product shaped by a diverse customer base over bespoke projects for one large customer; it says custom systems can take years to build, may not work, and may solve a problem the customer no longer has.
- Its product organization is built for rapid iteration: manufacturing changes every two weeks, code ships daily or in major biweekly releases, and product works alongside customer success and implementation, with research closely tied to customer problems so new capabilities can reach the field faster.
- Tutor sells on operational value rather than AI messaging, pitching robots at $16 an hour; it reports a 40–60-day average from first call to signed contract and about 40 days from contract to deployment. The buyer is often a COO or general manager while factory workers are the users, so naming robots is one way Tutor builds user ownership and rapport.
Hiten Shah
Coding agents make it easier to discard a working first version, supporting a more disposable approach to early product implementation.
ProductManagementJobs
A commenter said their organization would soon have a product manager vacancy and invited interested people to DM, but provided no employer, timing, or role-level details.