ai-mailer/Specification.md

Specification for Email Auto-Responder Service in Golang

1. Overview

This software is designed to automatically process incoming emails by:

• Fetching emails from an IMAP server on a configurable schedule.
• Fetching HTML content from a set of configurable URLs, without following any links.
• Using the OpenRouter AI API—with a configurable model—to generate a reply that uses both the email’s content and the website context.
• Saving the AI-generated reply as a draft in a designated IMAP folder.

The service runs continuously and processes emails sequentially, ensuring that each email is handled only once.

2. Functional Requirements

2.1 Email Processing

• Email Fetching:

  • Connect to an IMAP server using provided credentials.
  • Poll the server every x minutes (configurable via a config file).
  • Retrieve new emails from a specified mailbox (e.g., INBOX) that have not yet been processed.

• Uniqueness Handling:

  • Use IMAP’s built-in flags or markers (e.g., a custom flag) to mark emails as processed so that no email is processed more than once.

• Draft Creation:

  • Generate a reply based on AI output and save it as a draft in the designated IMAP Draft folder.
  • Format the draft email as a standard email (similar to Outlook formatting).

2.2 Context URL Fetching

• Configurable URLs:

  • The software will read a list of context URLs from the configuration file.
  • Each URL’s HTML content will be fetched using an HTTP client.
  • The fetched content will be used as context for the AI generation.

• No Link Following:

  • When retrieving the HTML content, the software should not follow or process any hyperlinks found within the content.

2.3 AI Integration

• Input Composition:

  • Combine the content of the email with the HTML content fetched from the configurable URLs.
  • Send this combined input to the OpenRouter AI API.

• Model Selection:

  • The AI model used for generating the reply should be selectable via a configuration setting.
  • The model identifier (or name) is passed as a parameter to the OpenRouter AI API call.

• Output:

  • Receive the generated reply that addresses the email’s content with the provided context.
  • Allow configuration of additional AI parameters (e.g., temperature, max tokens) through the config file.

3. Non-Functional Requirements

3.1 Configuration

• Config File:

  • All parameters, including IMAP server details, polling interval, list of context URLs, API keys, AI model and parameters, and logging preferences, must be settable via an external configuration file.

• Security:

  • Secure handling of sensitive data such as IMAP credentials and API keys (e.g., via environment variables or secure configuration management).
  • Ensure all network communications (IMAP over TLS and HTTPS for website/API calls) use secure protocols.

3.2 Error Handling & Logging

• Error Handling:

  • Implement robust error handling at every stage: IMAP connection, context URL fetching, AI integration, and draft creation.
  • Utilize retry mechanisms with exponential backoff for transient failures (e.g., network issues).

• Logging:

  • Provide detailed logging with configurable log levels (e.g., info, debug, error).
  • Log significant events such as successful email processing, context fetching outcomes, AI responses, errors, and retries.

3.3 Concurrency and Performance

• Sequential Processing:

  • Process emails sequentially to simplify the design, given the expected load.

• Long-Lived Service:

  • The software should operate continuously, handling graceful startup, shutdown, and configuration reloads as needed.

4. System Architecture

4.1 Modules/Components

1. Configuration Manager:

   • Loads and parses the configuration file.
   • Provides configuration data (e.g., IMAP details, context URLs, AI parameters, model selection) to other components.

2. IMAP Client Module:

   • Manages connections to the IMAP server.
   • Fetches new emails from the designated mailbox.
   • Marks emails as processed using IMAP flags or markers.
   • Saves generated replies to the Draft folder.

3. Context URL Fetcher Module:

   • Reads a list of URLs from the configuration.
   • Uses an HTTP client to fetch HTML content for each URL.
   • Ensures that no further links are followed during content retrieval.
   • Processes and optionally sanitizes the HTML for use as context.

4. AI Processor Module:

   • Combines the email content with the fetched context HTML.
   • Sends the combined content to the OpenRouter AI API with parameters (including the selected model and other settings) as specified in the configuration.
   • Receives and parses the AI-generated response.

5. Processing Controller:

   • Orchestrates the overall workflow.
   • Initiates email polling at the configured interval.
   • For each unprocessed email, performs:
     • Fetching the email.
     • Fetching context from all configurable URLs.
     • Invoking the AI processor.
     • Formatting the AI-generated reply as a standard email.
     • Saving the reply as a draft.
     • Marking the original email as processed.

6. Logging and Monitoring Module:

   • Captures and records all operations, errors, and significant events.
   • Provides health and status reports for debugging and operational oversight.

4.2 Data Flow

1. Startup:

   • Load configuration and initialize all modules.
   • Establish a connection to the IMAP server.

2. Polling Loop:

   • At every configured interval, poll for new emails.
   • For each new email:
     1. Retrieve email content.
     2. Fetch HTML content from each URL specified in the configuration (without following any links).
     3. Combine email content and context HTML.
     4. Send the combined data to the OpenRouter AI API, using the selected model.
     5. Receive and format the generated reply.
     6. Save the reply in the Draft folder.
     7. Mark the email as processed.

3. Error Handling:

   • Log and handle errors at each stage.
   • Use retry strategies where applicable to handle transient failures.

5. Deployment & Maintenance

• Deployment:

  • Build the Golang service into a standalone binary.
  • Deploy the binary along with its configuration file to the target environment.
  • Ensure that secure connections (IMAP over TLS, HTTPS for API and URL fetching) are configured properly.

• Maintenance:

  • Monitor logs and error reports for issues.
  • Update configuration settings as needed without code modifications.
  • Implement graceful shutdown and restart mechanisms to avoid processing duplicates.

6. Future Considerations

• Scaling:

  • Although sequential processing is used initially, consider refactoring to support parallel processing if email volume increases.

• Enhanced Content Processing:

  • Improve HTML content parsing to better extract relevant context or perform additional sanitization if required.

• Extensibility:

  • Allow dynamic updates to the list of context URLs or integration with other AI models as future enhancements.