Competitor Spy Tool

Overview

The Competitor Spy Tool automates continuous monitoring of competitor websites, pricing structures, content updates, and search engine rankings. This skill integrates with SerpAPI for SERP tracking, web_search for content discovery, Slack for real-time alerts, and Google Sheets for historical data logging.

Why it matters: Competitive intelligence is mission-critical. Manual monitoring across 5+ competitors is time-consuming and error-prone. This tool eliminates guesswork by automating daily scans, flagging price changes within 24 hours, tracking new blog posts, and monitoring keyword position shifts—enabling you to react faster than competitors.

Perfect for: SaaS founders, e-commerce managers, digital agencies, content strategists, and growth teams who need to stay ahead of market movements.

---

Quick Start

Try these prompts immediately:

CODEBLOCK0

CODEBLOCK1

CODEBLOCK2

CODEBLOCK3

CODEBLOCK4

---

Capabilities

1. Pricing Monitoring

Automatically scrapes and tracks competitor pricing pages, detects changes, and calculates percentage deltas. Stores historical pricing data in a spreadsheet for trend analysis.

Example usage:
CODEBLOCK5

2. Content Discovery & Tracking

Monitors RSS feeds, blog indexes, and sitemap updates to detect new content within hours. Categorizes posts by topic (pricing, features, case studies, thought leadership) and extracts key metadata.

Example usage:
CODEBLOCK6

3. Keyword Position Tracking

Uses SerpAPI to track your competitors' keyword rankings across 50+ target keywords. Shows position changes, search volume, and estimated traffic for each keyword.

Example usage:
CODEBLOCK7

4. Landing Page Monitoring

Captures screenshots and stores HTML snapshots of competitor landing pages. Detects copy changes, CTA button text modifications, headline shifts, and visual redesigns.

Example usage:
CODEBLOCK8

5. Competitive Intelligence Reporting

Generates weekly/monthly reports with:

• - Pricing trend analysis
• Content volume and topic distribution
• Keyword ranking movements
• Market share signals
• Actionable gaps and opportunities

---

Configuration

Required Environment Variables

CODEBLOCK9

Setup Instructions

1. 1. Get API Keys:

- SerpAPI: Sign up at serpapi.com, create API key - Slack: Create webhook at api.slack.com/apps - Google Sheets: Enable API in Google Cloud Console

1. 2. Add Competitor Domains:

CODEBLOCK10

1. 3. Initialize Google Sheet:

- Create sheet with columns: Date, Competitor, Metric, Old Value, New Value, Change % - Share with service account email

1. 4. Test Connection:

   curl -X POST $SLACK_WEBHOOK_URL \
     -H 'Content-Type: application/json' \
     -d '{"text":"Competitor Spy Tool initialized ✅"}'
   

---

Example Outputs

Slack Alert (Price Change Detected)

CODEBLOCK12

Content Discovery Report

CODEBLOCK13

Keyword Ranking Report

CODEBLOCK14

Weekly Intelligence Summary

📈 WEEKLY COMPETITIVE INTELLIGENCE
Period: Jan 8-14, 2024

PRICING MOVEMENTS:
  • 2 competitors raised prices (avg +8%)
  • 1 competitor introduced annual discount
  • Enterprise tier pricing now $599-999 (range: 500-1200)

CONTENT VELOCITY:
  • Avg posts/competitor: 6.3/week
  • Top topics: AI features (35%), case studies (25%)
  • Your gap: Limited thought leadership content

KEYWORD SHIFTS:
  • 15 keywords tracked
  • Avg position change: -0.2 (slight improvement)
  • New keyword targets emerging: "generative AI marketing"

RECOMMENDED ACTIONS:
  1. Publish 2-3 thought leadership posts on AI
  2. Consider enterprise pricing adjustment
  3. Target "generative AI marketing" keyword cluster

---

Tips & Best Practices

1. Set Smart Alert Thresholds

• - Price changes: 5-10% (lower for commodities, higher for SaaS)
• Content: Alert on product announcements, not every blog post
• Keywords: Flag position changes >3 spots, not minor fluctuations

2. Organize by Competitive Tier

Monitor direct competitors (same market, price) separately from adjacent players. Direct competitors warrant daily checks; adjacent players, weekly.

CODEBLOCK16

3. Create Keyword Clusters

Group keywords by intent (pricing, features, use cases) to spot trends faster.

CODEBLOCK17

4. Establish Baseline Data

Run initial scans for 2 weeks to establish normal patterns before setting alerts. This prevents false positives from minor page updates.

5. Cross-Reference with Your Metrics

Correlate competitor pricing changes with your conversion rates, churn, and customer feedback. Price wars often signal market saturation.

6. Automate Report Distribution

Schedule weekly reports to Slack #competitive-intel channel. Include executive summary + detailed metrics for strategy team.

---

Safety & Guardrails

What This Skill Will NOT Do

• - Scrape protected content: This tool respects robots.txt and Terms of Service. It will not bypass authentication, paywalls, or CAPTCHA challenges.
• Access private data: No scraping of customer data, internal documents, or confidential information.
• Violate GDPR/CCPA: No personal data collection. Monitoring is limited to public-facing pages only.
• Impersonate users: All requests include proper User-Agent headers and identify as a monitoring bot.
• Exceed rate limits: Built-in throttling respects server load (max 10 requests/minute per domain).

Ethical Boundaries

This tool monitors public, published content only:

• - ✅ Public pricing pages
• ✅ Published blog posts and RSS feeds
• ✅ Public search engine rankings
• ✅ Landing page copy and screenshots
• ❌ Customer databases or user accounts
• ❌ Confidential internal documents
• ❌ Password-protected areas

Legal Compliance

• - Ensure your use complies with local laws and competitor ToS
• Document your monitoring for audit trails
• Do not share competitor data publicly without attribution
• Use intelligence for strategic decisions, not misleading marketing claims

---

Troubleshooting

"API Rate Limit Exceeded"

Problem: SerpAPI or competitor servers are rate-limiting requests. Solution:

• - Reduce check frequency from daily to every 2 days
• Increase delay between requests: delay: 5000ms in config
• Upgrade SerpAPI plan for higher limits

"Slack Webhook URL Invalid"

Problem: Webhook URL is expired or incorrectly formatted. Solution: CODEBLOCK18

"No Price Changes Detected (False Negative)"

Problem: Competitor uses dynamic pricing, JavaScript rendering, or session-based prices. Solution:

• - Enable Browserless rendering: INLINECODE2
• Add CSS selectors for price elements: INLINECODE3
• Manually verify on competitor site; may be A/B testing

"Google Sheets Not Updating"

Problem: Service account lacks write permissions. Solution: CODEBLOCK19

"Competitor Domain Returns 403/429"

Problem: Server is blocking automated requests. Solution:

• - Add rotating User-Agent headers
• Use residential proxy service (Bright Data, Smartproxy)
• Increase delay between requests
• Contact competitor's support if monitoring is legitimate

FAQ

Q: How often should I check competitors?
A: Daily for direct competitors, weekly for adjacent players, monthly for emerging threats.

Q: Can I monitor 50+ competitors?
A: Yes, but scale API costs. Consider tiering: top 5 daily, next 15 weekly, others monthly.

Q: What if a competitor blocks my IP?
A: Use a proxy rotation service or contact them directly—legitimate competitive monitoring is defensible.

Q: How do I export data for analysis?
A: All data syncs to Google Sheets. Export as CSV/Excel for further analysis in Tableau, Looker, or custom dashboards.

Q: Is this legal?
A: Yes, monitoring public content is legal. Ensure you're not violating ToS, accessing protected areas, or using data deceptively.

---

Integration Examples

Slack Workflow

CODEBLOCK20

Google Sheets Historical Tracking

CODEBLOCK21

WordPress Blog Integration

CODEBLOCK22

Zapier Automation

Competitor price drop → Create Asana task → Notify sales team → Log to CRM

---

Support & Updates

For issues, feature requests, or contributions: GitHub Issues

Version History:

• - 1.0.0 (Current): Initial release with pricing, content, and keyword monitoring

---

Tags: competitive-intelligence, web-scraping, market-research, pricing-monitoring, seo-tracking, automation, saas, ecommerce