predictor/README.md

# Predictor Service

A Go-based weather prediction service that downloads and processes GRIB files for wind vector data extraction.

## Features

- **GRIB File Processing**: Downloads and processes GRIB weather data files
- **Wind Vector Extraction**: Extracts wind vector data for given coordinates and time
- **Redis Caching**: Caches extraction results for improved performance
- **Distributed Locking**: Uses Redis-based distributed locks for safe concurrent downloads
- **Scheduled Updates**: Automatic GRIB file updates via configurable scheduler
- **REST API**: HTTP API for data extraction and service management
- **Modular Architecture**: Clean separation of concerns with dependency injection

## Architecture

The service follows a modular architecture with clear separation of concerns:

- **Service Layer**: Business logic and orchestration
- **GRIB Package**: GRIB file processing and data extraction
- **Redis Package**: Caching and distributed locking
- **Scheduler Package**: Job scheduling and execution
- **Transport Layer**: HTTP API handling
- **Jobs**: Background tasks (GRIB updates, etc.)

## Quick Start with Docker

### Prerequisites

- Docker
- Docker Compose

### Running the Service

1. **Clone the repository:**
   ```bash
   git clone <repository-url>
   cd predictor
   ```

2. **Start the services:**
   ```bash
   # Production
   docker-compose up -d
   
   # Development (with volume mounts)
   docker-compose -f docker-compose.dev.yml up -d
   ```

3. **Check service status:**
   ```bash
   docker-compose ps
   ```

4. **View logs:**
   ```bash
   # All services
   docker-compose logs -f
   
   # Specific service
   docker-compose logs -f predictor
   docker-compose logs -f redis
   ```

### Using Make Commands

The project includes a Makefile for common operations:

```bash
# Build and start services
make up-build

# View logs
make logs

# Stop services
make down

# Clean up everything
make clean

# Show all available commands
make help
```

## Configuration

The service is configured via environment variables with the prefix `GSN_PREDICTOR_`:

### GRIB Configuration
- `GSN_PREDICTOR_GRIB_DIR`: Directory for GRIB files (default: `/tmp/grib`)
- `GSN_PREDICTOR_GRIB_TTL`: GRIB file TTL (default: `24h`)
- `GSN_PREDICTOR_GRIB_CACHE_TTL`: Cache TTL (default: `1h`)
- `GSN_PREDICTOR_GRIB_PARALLEL`: Parallel download workers (default: `4`)
- `GSN_PREDICTOR_GRIB_TIMEOUT`: Download timeout (default: `30s`)
- `GSN_PREDICTOR_GRIB_DATASET_URL`: GRIB data source URL

### Redis Configuration
- `GSN_PREDICTOR_REDIS_HOST`: Redis host (default: `localhost`)
- `GSN_PREDICTOR_REDIS_PORT`: Redis port (default: `6379`)
- `GSN_PREDICTOR_REDIS_PASSWORD`: Redis password (default: empty)
- `GSN_PREDICTOR_REDIS_DB`: Redis database (default: `0`)

### Scheduler Configuration
- `GSN_PREDICTOR_SCHEDULER_ENABLED`: Enable scheduler (default: `true`)

### GRIB Updater Job Configuration
- `GSN_PREDICTOR_GRIB_UPDATER_INTERVAL`: Update interval (default: `6h`)
- `GSN_PREDICTOR_GRIB_UPDATER_TIMEOUT`: Update timeout (default: `45m`)

### REST Transport Configuration
- `GSN_PREDICTOR_REST_HOST`: HTTP host (default: `0.0.0.0`)
- `GSN_PREDICTOR_REST_PORT`: HTTP port (default: `8080`)
- `GSN_PREDICTOR_REST_READ_TIMEOUT`: Read timeout (default: `30s`)
- `GSN_PREDICTOR_REST_WRITE_TIMEOUT`: Write timeout (default: `30s`)
- `GSN_PREDICTOR_REST_IDLE_TIMEOUT`: Idle timeout (default: `60s`)

## API Endpoints

The service exposes a REST API for wind data extraction:

- `GET /health` - Health check endpoint
- `POST /predict` - Extract wind data for given coordinates and time

### Example API Usage

```bash
# Health check
curl http://localhost:8080/health

# Extract wind data
curl -X POST http://localhost:8080/predict \
  -H "Content-Type: application/json" \
  -d '{
    "latitude": 40.7128,
    "longitude": -74.0060,
    "altitude": 100,
    "timestamp": "2024-01-15T12:00:00Z"
  }'
```

## Development

### Local Development

1. **Install dependencies:**
   ```bash
   go mod download
   ```

2. **Start Redis:**
   ```bash
   docker run -d --name redis -p 6379:6379 redis:7.2-alpine
   ```

3. **Set environment variables:**
   ```bash
   cd cmd/api
   source .env
   ```

4. **Run the service:**
   ```bash
   go run .
   ```

### Building Locally

```bash
# Build the binary
make build-local

# Run tests
make test

# Format code
make fmt

# Lint code
make lint
```

### Docker Development

For development with hot reloading:

```bash
# Start development environment
docker-compose -f docker-compose.dev.yml up -d

# View logs
docker-compose -f docker-compose.dev.yml logs -f predictor
```

## Docker Best Practices

The Dockerfile follows Go best practices:

- **Multi-stage build**: Separate builder and runtime stages
- **Non-root user**: Runs as non-root user for security
- **Minimal runtime**: Uses Alpine Linux for smaller image size
- **Health checks**: Built-in health monitoring
- **Optimized layers**: Efficient layer caching
- **Security**: No unnecessary packages or permissions

## Monitoring and Health Checks

The service includes built-in health checks:

- **Application health**: HTTP endpoint at `/health`
- **Docker health**: Container health check with wget
- **Redis health**: Redis ping health check
- **Service dependencies**: Proper startup order with health checks

## Troubleshooting

### Common Issues

1. **Redis connection refused:**
   - Ensure Redis is running: `docker-compose ps`
   - Check Redis logs: `docker-compose logs redis`
   - Verify network connectivity

2. **GRIB download failures:**
   - Check internet connectivity
   - Verify GRIB data source URL
   - Check disk space in `/tmp/grib`

3. **Service not starting:**
   - Check logs: `docker-compose logs predictor`
   - Verify environment variables
   - Check port conflicts

### Debug Commands

```bash
# Execute shell in container
docker-compose exec predictor sh

# Check Redis connectivity
docker-compose exec redis redis-cli ping

# View container resources
docker stats

# Check network connectivity
docker-compose exec predictor wget -O- http://redis:6379
```

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests
5. Run tests and linting
6. Submit a pull request

## License

[Add your license information here]