AI Model Wiki

    Skip to main content
    15 min
    Lesson 5 of 5

    Common Pitfalls & How to Avoid Them

    From the course Linear AI Engineering PM

    Linear AI delivers real value, but teams often make avoidable mistakes during rollout. Here are the most common pitfalls and the fixes.

    Pitfall 1: Skipping the Calibration Period

    What happens: Teams enable auto-triage, see a few wrong assignments in the first week, and disable it.

    Why it fails: Linear's AI learns from your team's correction patterns. The first 2–3 weeks produce lower accuracy because the model has not yet seen enough of your team's conventions.

    The fix: Commit to a 3-week calibration window. For each incorrect auto-triage result, correct it in Linear rather than overriding silently. Corrections are training signals — the more you correct explicitly, the faster accuracy improves. Set a calendar reminder to review triage quality on day 21 before making any decision to turn it off.

    Pitfall 2: Using AI Issue Creation as a Shortcut for Vague Requirements

    What happens: Engineers write one-line issue descriptions and rely on AI to flesh them out. AI-generated issues are structurally complete but contain hallucinated acceptance criteria that no one verifies.

    Why it fails: AI generates plausible-sounding acceptance criteria from the topic — not from actual product requirements. Unverified AI criteria ship as assumptions.

    The fix: Treat AI issue creation as a template, not a source of truth. Use it to generate the structure (description, acceptance criteria shell, sub-issues), then require human review of each criterion before the issue enters a cycle. A simple Linear workflow — add a "Criteria Verified" label before an issue can move to In Progress — creates the right checkpoint.

    Pitfall 3: Ignoring Cycle Intelligence Warnings

    What happens: Cycle completion probability drops below 70%, the team notices, and no one acts because "it usually works out."

    Why it fails: The AI signal has a lead time advantage. By day 8 of a 10-day cycle, it is too late to make meaningful scope changes. The warning is useful on day 2–3 when there is still time to defer issues without disrupting the cycle.

    The fix: Build a ritual — every second morning of a cycle, the PM or EM reviews cycle intelligence for 5 minutes. Any probability below 75% triggers a quick async discussion in Slack: which 1–2 issues can move to next cycle? This takes less than 10 minutes and avoids the last-day scramble.

    Pitfall 4: Over-Automating Stakeholder Updates

    What happens: Teams enable AI project updates and auto-share them to Slack or email without review. Stakeholders receive a technically accurate but context-free summary that creates confusion or misaligned expectations.

    Why it fails: AI update generation does not know which items are politically sensitive, which blockers have already been communicated verbally, or what narrative framing the stakeholder expects.

    The fix: Use AI updates as a first draft, not a send-ready document. Add a 3-minute human review layer: check for missing context, adjust tone, and add one sentence of "what this means for the business" at the top. The AI draft still saves 80% of writing time while the human layer prevents misalignment.

    Pitfall 5: Treating Duplicate Detection as Infallible

    What happens: An engineer sees "possible duplicate" on a new issue and closes it without investigation. The original issue had a different root cause.

    The fix: Duplicate detection surfaces similarity — it does not confirm identical root causes. Require engineers to open both issues and compare before merging. A simple comment thread in Linear ("confirmed duplicate because [reason]") creates an audit trail and improves team trust in the feature over time.

    Key Takeaways

    • •Commit to a 3-week calibration window before evaluating auto-triage — early inaccuracies are training signals, not failures
    • •AI issue creation generates structure, not truth — require human review of all acceptance criteria before a cycle starts
    • •Check cycle intelligence on day 2–3 when there is still time to act; day-8 warnings arrive too late to matter
    • •AI project updates need a 3-minute human review to add business context and prevent stakeholder misalignment
    0 of 5 lessons marked complete

    Progress is kept in this browser only.