farmos-observations

# FarmOS Observations AI-powered quick capture system — field observations, photos, voice notes, and issue reports. ## When to Use This **What this skill handles:** Field observations -- pest/disease/weed reports, crop condition notes, weather damage, soil issues, equipment problems spotted in the field, and photo-based scouting captures. **Trigger phrases:** "found [pest/weed/disease] in field X", "beans look rough", "something is wrong with field 12", "create an observation", "log this problem", "any observations today?", "what has been reported in field X?" **What this does NOT handle:** Equipment maintenance scheduling or fleet status (use farmos-equipment), task/work order creation (use farmos-tasks -- but the bot will offer to create a work order after logging an observation), weather forecasts or spray conditions (use farmos-weather). **Minimum viable input:** Any mention of something observed in the field. "Beans look bad" is enough -- the bot will ask smart follow-ups. ## Data Completeness 1. **The `/api/integration/dashboard` endpoint is for summary stats only** — observation counts and pending reviews. Do NOT use it to list individual observations. 2. **For listing observations**, use `GET /api/observations` with appropriate filters. This endpoint is paginated — use `limit` parameter and note the total. 3. **Always state the count**: "Found 7 observations this week in field 12" — not just a list without context. 4. **If results seem low**, flag it: "Only seeing 2 observations this week — that may be incomplete, or the observations service may be having issues." 5. **If the service is down**, say so plainly. Don't present empty results as "no observations." ## API Base http://100.102.77.110:8008 **Note:** The observations backend may have stability issues (restart loops reported). If endpoints don't respond, report that the observations service appears to be down. ## Integration Endpoints (No Auth) ### Dashboard GET /api/integration/dashboard Returns: Observation counts, recent activity, pending reviews. ## Authenticated Endpoints (JWT Required) ### Authentication This skill accesses protected FarmOS endpoints that require a JWT token. **To get a token:** ```bash TOKEN=$(~/clawd/scripts/farmos-auth.sh manager) ``` **To use the token:** ```bash curl -H "Authorization: Bearer $TOKEN" http://100.102.77.110:8008/api/endpoint ``` **Token expiry:** Tokens last 15 minutes. If you get a 401 response, request a new token. ### List Observations GET /api/observations?limit=10&field_id=12 Authorization: Bearer {token} ### Observation Detail GET /api/observations/{id} Authorization: Bearer {token} Returns: Full observation with AI analysis results, extracted entities, urgency score, and any created actions (tasks, maintenance records). ### Create Observation POST /api/observations Authorization: Bearer {token} Content-Type: multipart/form-data Form fields: - `observation_type` (required) — pest, disease, weed, weather_damage, equipment_issue, soil, crop_condition, other - `description` (required) — Text description of what was observed - `severity` (optional) — low, medium, high (default: medium) - `field_id` (optional) — Numeric field ID - `equipment_id` (optional) — Numeric equipment ID - `photo` (optional) — Image file attachment Example using curl: ```bash curl -X POST http://100.102.77.110:8008/api/observations \ -H "Authorization: Bearer $TOKEN" \ -F "observation_type=weed" \ -F "description=Found waterhemp in northeast corner near waterway" \ -F "severity=high" \ -F "field_id=22" \ -F "photo=@/path/to/photo.jpg" ``` **When crew reports a problem in #field-support or #field-ops, offer to create an observation.** Extract as much detail as you can from the message (field, observation type, severity), then create the observation. ## Usage Notes - Observations include urgency scores (1-10). Flag anything 7+ immediately. - AI processing classifies type and extracts equipment/field/crop references. - Observations may create tasks (via Task Manager) or maintenance records (via Equipment). - If the service is down, let the user know and suggest they log the observation manually. - The observation intake system is designed for photos from the field — the bot should be ready to accept image messages and route them here. - **Proactive observation creation:** When crew mentions issues in channel conversations ("got a bunch of weeds in field 12", "header making a weird noise"), offer to log it as an observation. Don't create silently — ask first. - **Equipment observations:** If the observation involves equipment, include `equipment_id` when creating. This helps track equipment-specific recurring issues. --- ## Smart Observation Detection When a user reports something that sounds like a field observation, auto-detect as much as you can from the message before asking questions. ### What to Detect **Field identification:** - Explicit: "field 12", "F12", "the 12" - By name: "the Byrd farm", "Kruckeberg", "home place" -- match to known field names - From channel context: if the conversation was already about a specific field, carry that forward - From user location: if they mention "the field I am in" or "out here", check recent context **Observation type** -- see the Observation Type Detection table below. **Severity** -- see the Severity Detection table below. **Specific pest/disease/weed identification:** - Common Indiana pests: western corn rootworm, Japanese beetle, corn earworm, soybean aphid, bean leaf beetle, armyworm, black cutworm, stink bug - Common diseases: tar spot, gray leaf spot, northern leaf blight, sudden death syndrome, white mold, frogeye leaf spot, Goss wilt, anthracnose - Common weeds: waterhemp, marestail (horseweed), giant ragweed, common ragweed, Palmer amaranth, lambsquarters, foxtail, velvetleaf, morningglory - If the reporter uses a colloquial name, map it: "buttonweed" -> common buttonweed, "volunteer corn" -> note as weed/volunteer **Equipment reference:** - By name/number: "the 8250", "the Kinze", "the planter", "sprayer" - By implication: "header won't raise" implies the combine (likely 8250) **Location within field:** - Cardinal directions: "northeast corner", "south end" - Landmarks: "near the waterway", "along the tree line", "by the road", "headlands", "terrace" - Coverage: "whole field", "scattered", "in patches", "one spot" --- ## Observation Type Detection | Keywords / Signals | Observation Type | |--------------------|-----------------| | bug, insect, aphid, rootworm, armyworm, beetle, cutworm, earworm, stink bug, larva, grub | pest | | tar spot, gray leaf spot, northern leaf blight, rust, rot, blight, lesion, spots on leaves, mold, wilt, SDS, anthracnose, frogeye | disease | | waterhemp, marestail, ragweed, foxtail, lambsquarters, Palmer, volunteer corn, weeds, escapes, resistance | weed | | hail, wind damage, flood, frost, drought stress, storm, ice, lightning, washout, ponding | weather_damage | | broken, leaking, stuck, noise, won't start, overheating, vibration, warning light, hydraulic, flat tire | equipment_issue | | compaction, erosion, drainage, wet spots, tile, washout, ruts, soil test, pH | soil | | stand count, emergence, color, lodging, population, uneven, stunted, yellowing, purpling, canopy | crop_condition | **If multiple types match** (e.g., "yellowing leaves with spots" could be disease or crop_condition), pick the more specific one (disease in that case). If genuinely ambiguous, ask: "Is this more of a disease issue or general crop condition?" --- ## Severity Detection | Language Signals | Severity | |-----------------|----------| | "bad", "terrible", "everywhere", "whole field", "never seen this before", "worst I have seen", "out of control", "lost cause" | high | | "some", "moderate", "spreading", "getting worse", "more than last week", "quite a bit", "a lot of" | medium | | "a few", "small patch", "just noticed", "isolated", "one spot", "not too bad", "just starting" | low | **Default to medium** if the language is neutral or you cannot determine severity. Never guess high -- ask. --- ## Follow-Up Questions (Smart) When the report is sparse, ask targeted follow-up questions. **Maximum 2-3 questions per interaction -- do not interrogate.** ### Question Bank (pick the most useful ones) - **Scope:** "How widespread is it? Just that spot or across the field?" - **Adjacent fields:** "Have you seen this in adjacent fields?" - **Percentage:** "Roughly what percentage of the field is affected?" - **Photos:** "Any photos? A picture would help me nail down exactly what this is." - **Treatment history:** "Was this field treated with anything recently?" - **Recurrence:** "Is this the same issue you reported on [date]?" (only if a similar recent observation exists -- check observation history first) - **Timing:** "When did you first notice this?" - **Trend:** "Is it getting worse or holding steady?" ### Which Questions to Ask | Missing Info | Ask | |-------------|-----| | No field identified | "Which field are you in?" (always ask this -- cannot create without a field) | | No type identifiable | "What are you seeing exactly? Weeds? Bugs? Crop damage? Equipment problem?" | | Has type but no scope | "How widespread is it?" | | Weed/pest report | "Any photos?" (visual ID is high value for these) | | Sounds like recurring issue | "Is this the same issue from [date]?" | --- ## Reporter-Adaptive Behavior ### Detailed Reporters (agronomists, Brian, experienced crew) - They will say things like "Found western corn rootworm beetle damage in field 22, northeast corner, about 15% of the rows affected, adult count was 8 per plant" - **Auto-detect everything. Confirm the details. Create the observation.** - Skip redundant questions -- they have already given you what you need - Example response: "Got it -- rootworm beetle damage in F22, northeast corner, ~15% of rows, 8 adults/plant. Severity: high. Creating the observation now." ### Sparse Reporters (field crew, seasonal workers) - They will say things like "something's wrong with field 12" or "beans look bad" - **Ask 2-3 targeted follow-up questions. Do NOT interrogate.** - Pick the highest-value questions for the situation - After 2-3 questions, create the observation with whatever you have - Example exchange: - Crew: "Beans look rough in 12" - Bot: "What are you seeing -- weeds, bugs, disease, or just the crop looking off? And is it the whole field or just a section?" - Crew: "Weeds. Northeast side mostly." - Bot: "Got it. Creating an observation -- weed pressure in field 12, northeast section, medium severity. Want me to create a scouting work order for it too?" ### Key Rules - **Always offer before creating:** "Want me to create an observation for this?" Do NOT create silently -- unless the user has explicitly asked you to create one. - **Exception:** If the user directly says "Log this" or "Create an observation for...", that IS explicit permission -- go ahead and create. - **Never ask more than 3 follow-up questions.** After 3, create with what you have and note what was unknown. --- ## Post-Creation Actions After successfully creating an observation, offer related actions: - **Work order:** "Want me to create a work order for this?" (especially for weed/pest/equipment issues that need action) - **Adjacent field check:** "Should I check if adjacent fields have the same issue?" (useful for pest/disease/weed spread) - **Scouting task:** "Want me to schedule a follow-up scouting trip?" (for observations that need monitoring) - **Equipment maintenance:** If equipment_issue type, "Want me to log this against the equipment maintenance record?" Only offer 1-2 of the most relevant follow-up actions. Do not overwhelm the reporter with options. --- ## Urgency Escalation **Flag the operator immediately** (in addition to creating the observation) for: - **Crop damage** at high severity -- potential yield loss, Brian needs to know - **Equipment safety** -- hydraulic leak, structural failure, anything that could injure someone - **Chemical exposure** -- drift, spill, re-entry violation, any chemical safety concern - **Pest/disease outbreak** -- high severity pest or disease that could spread rapidly (tar spot, sudden death syndrome, heavy rootworm pressure) - **Weather damage** at high severity -- hail, flood, significant storm damage When escalating, send a concise alert: "URGENT: [type] reported in field [X] -- [one-line summary]. Severity: high. Observation #[id] created." Do NOT escalate routine observations (low/medium severity, isolated issues, normal scouting finds). --- ## Cross-Module Context After creating or reviewing observations, connect to other modules: **Observations → Pattern Detection:** - When a new observation is created, check for similar recent observations (same type, nearby fields, same week): "This is the third waterhemp observation this week across three different fields. Might be time for a blanket spray program rather than spot treatments." - When listing observations, group by pattern when multiple share type/field/timeframe: "3 disease observations this week, all in the east fields. Could be spreading." - Track escalation: "First report was low severity last Tuesday, now we're at high severity across 4 fields. This is moving fast." **Observations → Tasks:** - After creating an observation, check farmos-tasks for existing work orders related to this field or issue: "There's already a scouting task open for field 22 from yesterday — want me to add this observation to it?" - If no related task exists and the observation is actionable (pest, disease, weed at medium+ severity), offer to create one. - Connect observation patterns to task suggestions: "Third waterhemp sighting this week — want me to create a blanket spray task?" **Observations → Weather:** - Connect recent weather to observation context: "3 disease observations after last week's rain — moisture likely drove this." - When a weather_damage observation is created, pull the actual weather data: "You're reporting hail damage in field 14. Records show we got 1.2 inches with possible hail Tuesday evening." - Flag ongoing weather risk: "With more rain coming Thursday, expect this fungal pressure to continue." **Observations → Equipment:** - If an observation leads to a task that requires specific equipment, check equipment availability: "If you're going to spray for this waterhemp, the sprayer is available — 153 hours, no maintenance due." - For equipment_issue observations, cross-reference with farmos-equipment for that machine's maintenance history. Cross-reference when it adds context. A simple "log this observation" doesn't need a full cross-module sweep. But when patterns emerge or observations drive action, connect the dots. ## Image Understanding When a photo accompanies an observation (the image description will appear in your context as `[Image] Description: ...`), use it to enhance the observation record. ### Photo-Enhanced Detection **Use the image description to refine type and severity:** - Photo of lesions on leaves -- identify disease characteristics (shape, color, pattern, location on leaf) -- refine observation_type to "disease" and attempt specific ID - Photo of insects or insect damage -- identify species or damage pattern -- refine observation_type to "pest" - Photo of weeds -- identify species from leaf shape, growth habit, flower/seed head -- refine observation_type to "weed" - Photo of equipment in the field -- note the machine and any visible issues -- set observation_type to "equipment_issue" and include equipment_id - Photo of crop conditions -- note growth stage, color, stand count, uniformity -- refine observation_type to "crop_condition" - Photo of weather damage -- note damage pattern (hail bruising, wind lodging, flood line) -- set observation_type to "weather_damage" **Always include the image description in the observation `description` field.** Combine what the reporter said with what the photo shows: > "Reporter: Found some weird spots on the corn in field 12. Photo shows: rectangular tan lesions between leaf veins, approximately 1-3cm long, consistent with gray leaf spot (Cercospora zeae-maydis). Northeast section of field." ### Photo Quality Handling - **Clear photo:** Use it confidently to refine detection. State what you see and your assessment. - **Unclear/blurry/dark photo:** Say so honestly: "I can make out [what you can see] but the photo is too blurry/dark for a confident ID. Can you get a closer shot, or describe what you are seeing?" - **Photo does not match description:** If the photo shows something different from what the reporter described, mention it: "You mentioned weeds but the photo looks like it might be disease lesions -- can you clarify?" - **Multiple issues visible:** Note all of them: "I can see both waterhemp and what looks like tar spot lesions in this photo. Want me to create observations for both?" ### Photo Prompt for Sparse Reports When a reporter sends a text-only observation about something visual (pest, disease, weed, damage), and has NOT included a photo: - "Any photos? A picture would help me nail down exactly what this is." (already in the follow-up question bank) - Do NOT demand photos. A text description is always enough to create an observation. ### Attaching Photos to Observations When creating an observation via `POST /api/observations`, include the photo as the `photo` form field if a MediaPath is available in your context. The image gets archived with the observation record for future reference.