This commit completes steps 1-5 of the database migration plan, moving database responsibility from the webserver to the controller: 1. **Step 3 - Database API Endpoints**: Added comprehensive REST API endpoints in controller for all database operations (machines, extraction jobs, persons, companies, documents) with proper authentication and filtering 2. **Step 4 - Authentication Setup**: Implemented controller-webserver authentication with API keys and created detailed setup documentation (AUTHENTICATION_SETUP.md) 3. **Step 5 - Move Database Models**: Migrated webserver database models to controller and implemented webserver-specific database handler with CRUD operations for agents, tasks, logs, and settings Key changes: - Added query functions to controller/dbhandler.py for all database entities - Created webserver_dbhandler.py for webserver-specific tables - Implemented ControllerClient in webserver for API communication - Updated webserver routers to use controller API instead of direct database access - Added webserver migration SQL schema - Updated CI/CD workflows to reflect renamed webserver directory - Removed direct database dependencies from webserver (PostgreSQL, SQLAlchemy) - Updated docker-compose.yml for new architecture BREAKING CHANGE: Webserver no longer uses direct database access and requires controller API configuration (CONTROLLER_URL and CONTROLLER_API_KEY environment variables)
5.1 KiB
Authentication Setup Between Webserver and Controller
Overview
This document explains how to set up authentication between the Epistine webserver and controller as part of the database migration plan (Step 4).
Current Architecture
- Controller: Handles all database operations and provides REST API endpoints
- Webserver: Provides web interface and business logic, now uses controller API for database operations
- Authentication: Webserver authenticates with controller using API keys with scopes
Setup Steps
1. Start the Controller
First, ensure the controller is running:
cd controller
python app.py
The controller should start on http://localhost:8000 by default.
2. Create an API Key for the Webserver
The webserver needs an API key with appropriate scopes to access controller endpoints.
Using the Controller API:
# Use the master API key (from controller/.env) or create a new one
curl -X POST "http://localhost:8000/api/v1/createAPIKey" \
-H "FilesManager-API-Key: YOUR_MASTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"name": "webserver-api-key",
"created_by": "system",
"description": "API key for webserver to access controller",
"scopes": ["worker:read", "logs:read", "logs:write"]
}'
Required Scopes for Webserver:
worker:read- For reading machines/agentslogs:read- For reading logslogs:write- For creating logs- (Optional)
system:create_api_key- If webserver needs to create API keys
3. Configure the Webserver
Update the webserver's environment configuration:
-
Copy the example environment file:
cd epistine-webserver cp .env.example .env -
Edit the
.envfile and set:CONTROLLER_URL=http://localhost:8000 CONTROLLER_API_KEY=your-generated-api-key-here
4. Start the Webserver
cd epistine-webserver
docker-compose up
Or run directly:
cd epistine-webserver/backend
uvicorn app.main:app --reload --host 0.0.0.0 --port 8001
5. Verify Authentication
Run the authentication test script:
python simple_auth_test.py
Or manually test:
# Test controller health
curl http://localhost:8000/api/v1/health
# Test authentication (replace with your API key)
curl -H "FilesManager-API-Key: YOUR_API_KEY" \
http://localhost:8000/api/v1/machines
How It Works
Authentication Flow
- Webserver receives request from frontend or user
- Webserver calls controller API with API key in
FilesManager-API-Keyheader - Controller validates API key and checks scopes
- Controller processes request and returns data
- Webserver formats response and returns to client
Code Changes Made
-
Controller Client (
epistine-webserver/backend/app/controller_client.py):- HTTP client for communicating with controller
- Handles authentication headers
- Manages errors and timeouts
-
Updated Routers (
epistine-webserver/backend/app/routers/__init__.py):- All endpoints now use
ControllerClientinstead of direct database access - Proper error handling for controller communication
- Data mapping between controller and webserver models
- All endpoints now use
-
Environment Configuration (
epistine-webserver/.env.example):- Added
CONTROLLER_URLandCONTROLLER_API_KEYvariables
- Added
Troubleshooting
Common Issues
-
"Unauthorized access to controller" error
- Check that
CONTROLLER_API_KEYis set correctly - Verify the API key is valid and not revoked
- Ensure controller is running
- Check that
-
"Controller service unavailable" error
- Check controller health:
curl http://localhost:8000/api/v1/health - Verify
CONTROLLER_URLis correct - Check network connectivity
- Check controller health:
-
"Insufficient permissions" error
- The API key is missing required scopes
- Create a new API key with the required scopes
- See "Required Scopes for Webserver" section above
-
Data mapping issues
- Controller and webserver may use different data models
- Check the mapping logic in the router functions
- Update mapping as needed for your use case
Testing
Use the provided test scripts:
# Basic authentication test
python simple_auth_test.py
# Comprehensive test (requires controller running)
python test_authentication.py
Security Considerations
- API Key Storage: Store API keys in environment variables, not in code
- Key Rotation: Regularly rotate API keys for security
- Scope Minimization: Grant only necessary scopes to each API key
- Network Security: Use HTTPS in production, restrict network access
- Monitoring: Monitor authentication failures in controller logs
Next Steps
After authentication is set up:
- Move database models from webserver to controller (Step 5)
- Update Docker Compose for new architecture (Step 7)
- Add comprehensive testing (Step 8)
- Implement monitoring and logging (Step 9)