Lessons From Building a Chrome Extension With AI

The Extension That Taught Us Everything About Browser Constraints

ShelfMark started as a weekend project. The pitch was simple: a Chrome extension that uses AI to automatically organize your bookmarks into meaningful categories. No more manual folder management, no more “Unsorted” folders with 400 items, no more losing that article you saved three months ago because you can’t remember what you titled it or which folder you dropped it in.

That weekend project turned into a three-month engineering effort that touched every painful corner of Chrome extension development. Browser extensions occupy a unique space in software engineering — you’re building a product that lives inside someone else’s application, subject to their runtime constraints, their permission model, their review process, and their users’ deep (justified) suspicion of anything that touches their browsing data.

Here’s what we learned building ShelfMark, and why browser extensions with AI are simultaneously easier and harder than you’d expect.

Manifest V3: The Migration That Changed Everything

If you haven’t built a Chrome extension recently, the biggest shift is Manifest V3. Google has been migrating extensions from Manifest V2 to V3 since 2022, and as of 2025, V2 extensions are no longer accepted in the Chrome Web Store. If you’re starting a new extension today, V3 is your only option.

The headline change is the replacement of persistent background pages with service workers. In V2, your background script ran continuously — it could hold state in memory, maintain WebSocket connections, and respond to events without cold start latency. In V3, the service worker is ephemeral. It spins up on events, runs your handler, and terminates after 30 seconds of inactivity (extended to 5 minutes in some cases, but you can’t depend on that).

For ShelfMark, this fundamentally changed our architecture. The bookmark organization process — fetch all bookmarks, analyze them in batches with AI, propose a folder structure, apply changes — takes longer than 30 seconds for any non-trivial bookmark collection. We couldn’t run it in a single background script execution.

// The naive approach that DOESN'T work in MV3:
chrome.action.onClicked.addListener(async () => {
  const bookmarks = await chrome.bookmarks.getTree();
  const flatBookmarks = flattenTree(bookmarks); // Could be 1000+ items
  
  // This will die mid-execution when the service worker terminates
  for (const batch of chunk(flatBookmarks, 20)) {
    const categories = await analyzeWithAI(batch); // 3-5 seconds each
    await reorganizeBookmarks(categories);          // Chrome API calls
  }
  // Never reaches here for large bookmark collections
});

The solution was to decompose the process into discrete steps, persist state between steps using chrome.storage.local, and use chrome.alarms to wake the service worker for each step. Each alarm fires, the service worker wakes up, reads its state from storage, executes the next step, saves state, and schedules the next alarm.

// Step-based approach that survives service worker restarts
chrome.alarms.onAlarm.addListener(async (alarm) => {
  if (alarm.name !== 'shelfmark-organize') return;
  
  const state = await chrome.storage.local.get('organizeState');
  const currentStep = state.organizeState?.step || 'fetch';
  
  switch (currentStep) {
    case 'fetch': {
      const bookmarks = await chrome.bookmarks.getTree();
      const flat = flattenTree(bookmarks);
      await chrome.storage.local.set({
        organizeState: { 
          step: 'analyze', 
          bookmarks: flat, 
          batchIndex: 0, 
          results: [],
          totalBatches: Math.ceil(flat.length / 20),
          startedAt: Date.now()
        }
      });
      chrome.alarms.create('shelfmark-organize', { delayInMinutes: 0.1 });
      break;
    }
    case 'analyze': {
      const { bookmarks, batchIndex, results, totalBatches } = state.organizeState;
      const batch = bookmarks.slice(batchIndex * 20, (batchIndex + 1) * 20);
      
      if (batch.length === 0) {
        // All batches processed, move to proposal step
        await chrome.storage.local.set({
          organizeState: { step: 'propose', results }
        });
      } else {
        const analysis = await analyzeWithAI(batch);
        results.push(...analysis);
        await chrome.storage.local.set({
          organizeState: { 
            step: 'analyze', 
            bookmarks, 
            batchIndex: batchIndex + 1, 
            results,
            totalBatches 
          }
        });
        // Notify popup of progress
        chrome.runtime.sendMessage({
          type: 'progress',
          current: batchIndex + 1,
          total: totalBatches
        });
      }
      chrome.alarms.create('shelfmark-organize', { delayInMinutes: 0.1 });
      break;
    }
    case 'propose': {
      const { results } = state.organizeState;
      const proposal = generateFolderProposal(results);
      await chrome.storage.local.set({
        organizeState: { step: 'awaiting_approval', proposal }
      });
      // Show badge to indicate proposal is ready
      chrome.action.setBadgeText({ text: '!' });
      chrome.action.setBadgeBackgroundColor({ color: '#3B82F6' });
      break;
    }
    case 'apply': {
      const { proposal, approvedItems } = state.organizeState;
      await applyApprovedChanges(proposal, approvedItems);
      await chrome.storage.local.remove('organizeState');
      chrome.action.setBadgeText({ text: '' });
      break;
    }
  }
});

This pattern — state machine with persistent storage and alarm-based continuation — became the backbone of ShelfMark. It’s ugly compared to a straightforward async function, but it’s the only reliable way to run multi-step processes in MV3. The key insight: treat the service worker like a serverless function. It should be stateless, idempotent, and quick. State lives in storage, not in memory.

We also had to handle the case where the user closes Chrome mid-process. When the browser restarts, the service worker fires its onInstalled event. We check for an incomplete organizeState and resume from where we left off. Bookmarks haven’t changed (assuming no other extensions touched them), so resumption is safe.

The AI Integration Challenge

ShelfMark’s AI integration had a non-obvious constraint: where does the AI computation happen? We had three realistic options, each with significant tradeoffs.

Option 1: Call an external API (OpenAI, Claude) directly from the extension. Simplest implementation but fatal flaw: it requires either shipping an API key in the extension (a security disaster — anyone can extract it from the extension package within minutes) or requiring users to provide their own key (which limits adoption to technical users who have API accounts).

Option 2: Route through our own backend that calls the AI API. The backend holds the API key. Users authenticate with ShelfMark, not with OpenAI. We control costs, can rate limit abusive users, and can switch AI providers without updating the extension.

Option 3: Run a model locally in the browser. Chrome’s built-in AI APIs (Gemini Nano via the Prompt API) were experimental when we started and lacked the instruction-following capability we needed for reliable categorization. Running a WASM-based model (like Llama.cpp compiled to WebAssembly) was possible but the model size (~500MB+) and inference speed made it impractical for a lightweight extension.

We went with Option 2. ShelfMark calls our backend API, which handles AI inference and returns categorization results. This adds a network dependency but gives us full control over the model, prompt engineering, cost management, and the ability to improve categorization quality without shipping extension updates.

// Backend endpoint for bookmark analysis
app.post('/api/analyze-bookmarks', async (req, res) => {
  const { bookmarks, existingFolders, userPreferences } = req.body;
  
  const prompt = `You are organizing browser bookmarks into a clean folder hierarchy.
  
Existing folder structure (prefer reusing these):
${existingFolders.map(f => `- ${f.name} (${f.count} bookmarks)`).join('n')}

User preferences: ${userPreferences.style === 'broad' 
  ? 'Prefer fewer, broader categories' 
  : 'Prefer more specific, granular categories'}

Bookmarks to categorize:
${bookmarks.map(b => `- id:${b.id} "${b.title}" (${new URL(b.url).hostname})`).join('n')}

Rules:
1. Use existing folders when they fit — don't create duplicates
2. Create new folders only when no existing folder is appropriate
3. Folder names: 1-3 words, descriptive, Title Case
4. Each bookmark goes in exactly one folder
5. "Uncategorized" is valid for truly ambiguous items
6. Prioritize the URL domain for classification when the title is unclear

Return JSON: {
  "assignments": [{ "bookmarkId": "...", "folder": "..." }],
  "newFolders": ["..."]
}`;

  const result = await anthropic.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 2000,
    messages: [{ role: 'user', content: prompt }],
  });

  const parsed = parseAndValidateResponse(result);
  res.json(parsed);
});

The prompt went through many iterations. Early versions produced inconsistent folder names (“Development” vs “Dev” vs “Programming” for the same concept). Adding the existing folder list and the instruction to prefer reusing existing folders dramatically improved consistency. Including the URL hostname was another breakthrough — a bookmark titled “untitled” with a URL on github.com is clearly development-related, but the AI couldn’t know that from the title alone.

Privacy: The Make-or-Break Factor

Browser extensions have a trust problem, and rightfully so. The Chrome Web Store is full of extensions that harvest browsing data, inject ads, or sell user information to data brokers. Users are justifiably suspicious when an extension asks for bookmark access, and even more suspicious when that extension sends data to an external server.

We took privacy seriously from day one, not just ethically but architecturally:

• Minimal data transmission: We send bookmark titles and URLs to our backend. We do NOT send browsing history, open tabs, page content, cookies, or any other browser state. The extension’s manifest requests only bookmarks and storage permissions — nothing else.
• No data retention: The backend processes the analysis request and discards the input immediately. We log request metadata (timestamp, bookmark count, response time) for monitoring, but never the bookmark content itself. We don’t store bookmarks, don’t build user profiles, don’t train on user data.
• Transparent permissions: We request only bookmarks, storage, and alarms permissions. No tabs, no webNavigation, no history, no activeTab. The minimum viable permission set.
• Privacy policy with specifics: Not the generic “we respect your privacy” boilerplate. Our privacy policy enumerates exactly which data fields are transmitted, how long they persist (they don’t), and where the processing happens (our infrastructure, not third-party AI providers’ training pipelines).
• Offline-first design: ShelfMark works fully offline for manual bookmark organization — drag-and-drop folder management, search, bulk operations. AI categorization requires network access, but the extension is functional and useful without it.

The privacy stance also affected our AI prompt design. We instructed the model to categorize based solely on the URL structure and title, never to infer personal information about the user. The model doesn’t know who the user is — it just sees a list of titles and URLs with no identifying context.

The Chrome Web Store Review Process

Getting ShelfMark approved in the Chrome Web Store was its own multi-week adventure. The review process is opaque (you get a pass/fail with minimal explanation), slow (5-10 business days for initial review, 3-5 for updates), and strict about permission justifications.

Our first submission was rejected because we used the alarms permission without “adequate justification.” The reviewer wanted to know why a bookmark organizer needed to set alarms. We wrote a detailed explanation of the MV3 service worker limitation and how alarms enable multi-step processing of large bookmark collections. The second submission was approved.

Key lessons from the review process:

• Justify every permission in the store listing description. Don’t assume the reviewer understands why you need it. “We use the alarms permission to process large bookmark collections in batches because MV3 service workers terminate after 30 seconds of inactivity” is the level of specificity required.
• Avoid broad permissions. We initially requested activeTab for a planned feature that would extract the current page’s title for quick bookmarking. We dropped it because the core product didn’t need it, and adding unnecessary permissions complicates review and erodes user trust.
• Include a privacy policy. Required for any extension that transmits data. It should be specific about what data you collect, how it’s used, and how long it’s retained.
• Screenshots must show real functionality. Mock-ups, placeholder images, or screenshots from a different application get flagged. Use real screenshots of the actual extension running in Chrome.
• Test on Chrome stable, not Canary. If your extension uses APIs that only exist in Chrome Canary or Beta, it will be rejected. Only use APIs available in the current stable release.

UI in a 400×600 Pixel Popup

Extension popups are tiny. The default popup dimensions are constrained by Chrome, and while you can set explicit width and height, going beyond 800px wide or 600px tall feels intrusive. ShelfMark’s UI had to show a folder tree, a list of uncategorized bookmarks, AI suggestions, progress indicators, and action buttons — all in a space smaller than a mobile screen.

We used React with Tailwind CSS for the popup UI, built with Vite. The key UI decisions that made the constrained space work:

• Collapsible folder tree that shows only top-level folders by default, expanding on click. Folder counts are shown inline so users can identify large folders without expanding them.
• AI suggestions presented as a diff. Rather than silently moving bookmarks, ShelfMark shows proposed changes: “Move ‘PostgreSQL EXPLAIN guide’ from Unsorted to Development/Databases.” Users approve or reject each suggestion individually. This builds trust by keeping the user in control.
• Batch actions. For large reorganizations (100+ suggestions), reviewing one-by-one is tedious. We added “Approve All in Category” and “Reject All” buttons that let users make bulk decisions while retaining the ability to override specific items.
• Progress indicator for the analysis phase. Processing 500 bookmarks takes 30-60 seconds. Without clear progress feedback, users assume the extension is broken and close the popup (which doesn’t kill the background process but does create anxiety).
• Keyboard navigation. Power users manage bookmarks with keyboard shortcuts. Arrow keys navigate the suggestion list, Enter approves, Backspace rejects, Tab moves between sections. This makes the tiny popup surprisingly efficient to operate.

const SuggestionCard = ({ suggestion, onApprove, onReject }: Props) => (
  
 {
      if (e.key === 'Enter') onApprove();
      if (e.key === 'Backspace') onReject();
    }}>
    

      

        {suggestion.bookmark.title}
      
      

        {suggestion.fromFolder || 'Unsorted'}
        →
        {suggestion.toFolder}
      
    
    

      
        
      
      
        
      
    
  
);

Performance: Every Millisecond Counts

Extension popups need to open instantly. Users click the icon expecting immediate response. If the popup takes more than 200ms to render meaningful content, it feels broken. Web apps can get away with loading spinners. Extensions cannot.

Our initial React bundle was 180KB gzipped — acceptable for a web app, perceptibly slow for an extension popup that needs to paint in under 200ms. We optimized aggressively:

• Replaced heavy icon libraries (Heroicons, Lucide full package) with inline SVGs for the 8 icons we actually used. Saved 45KB.
• Lazy-loaded the settings page (accessed by ~5% of users) behind a dynamic React.lazy() import. Saved 22KB from the critical path.
• Preloaded bookmark data in the service worker so the popup reads from chrome.storage.local instead of calling the bookmarks API on open. The bookmarks API call takes 50-100ms for large collections; storage reads take under 5ms.
• Used Vite’s build optimization with manual chunk splitting to ensure the critical rendering path loads first.
• Final bundle: 42KB gzipped. Popup opens in under 100ms with content visible. No spinner, no skeleton screen, just content.

We also discovered that chrome.storage.local has a 10MB quota by default (expandable with the unlimitedStorage permission, which we deliberately avoid requesting). For users with 5,000+ bookmarks and accumulated suggestion history, we approach this limit. We implemented an LRU cache that prunes old suggestion history to stay within bounds.

Error Handling in a Disconnected Environment

Extensions operate in a more hostile environment than web apps. The network might be unavailable. The backend might be down. The service worker might restart mid-operation. Chrome might update and change API behavior. Every failure mode needs graceful handling because users can’t open DevTools on an extension to debug it — they just see a broken popup.

We categorize errors into three buckets: recoverable (retry with backoff), degradable (fall back to offline features), and fatal (show clear error message with recovery instructions). The backend being unreachable is degradable — the extension still works for manual bookmark management. A corrupt storage state is fatal — we show a “Reset ShelfMark” button that clears local state and starts fresh.

The most insidious errors are silent ones. If the AI returns a malformed response and our parser silently produces empty results, the user sees no suggestions and assumes they have no bookmarks to organize. We added validation for every AI response and explicit error states: “AI analysis returned no suggestions — this might mean your bookmarks are already well-organized, or there was a processing error. Try again?”

What We’d Build Differently Today

Chrome is shipping native AI APIs (Prompt API, Summarization API, Translation API) that weren’t available when we started. If we were building ShelfMark today, we’d use the Prompt API for categorization when available and fall back to our backend when it’s not. This would eliminate the privacy concern entirely for supported browsers — the AI runs locally, no data leaves the device.

We’d also consider building for Firefox simultaneously from day one. Firefox’s extension platform supports MV3 with some differences (they kept the persistent background page option as an alternative, which would have simplified our state machine significantly), and the Firefox Add-on Store review process is faster, more transparent, and provides actionable feedback. Launching on two browsers from day one doubles your addressable market with modest additional engineering effort.

Finally, we’d invest more in the onboarding experience. ShelfMark’s first-run flow currently asks users to click a button to start analysis. Many users expected it to work automatically on install. A guided walkthrough that explains what will happen, what data is sent, how to approve suggestions, and sets expectations for processing time would have improved our activation rate significantly.

The Broader Takeaway

Building a Chrome extension with AI forced us to think about constraints we rarely face in web development. Memory limits, execution time limits, strict permission justification, storage quotas, and extreme UI constraints made us better engineers. Every decision was a tradeoff — capability vs. permission scope, AI quality vs. privacy, feature richness vs. bundle size, user convenience vs. trust.

ShelfMark is a small product, but the lessons scale. The state machine pattern for long-running processes in constrained environments applies to serverless functions (Lambda’s 15-minute timeout), mobile apps (iOS background processing limits), and IoT devices. The privacy-first architecture — minimal data, no retention, transparent permissions — applies to any product that handles user data. The approval/rejection UX pattern — showing AI suggestions as proposals rather than fait accompli — applies to any AI product where trust is earned, not assumed.

If you’re considering building an AI-powered browser extension, do it. The constraints are real but manageable. The distribution channel (Chrome Web Store) puts you in front of millions of potential users without a marketing budget. And the problems worth solving in the browser are endless — bookmarks are just the beginning. ShelfMark barely scratches the surface of what’s possible when you combine the browser’s privileged access to user intent with AI’s ability to organize and interpret that intent.