| # Migrating from Celery to APScheduler | |
| This guide explains how to migrate from Celery to APScheduler in the Lin application. | |
| ## Why Migrate to APScheduler? | |
| 1. **Simplified Architecture**: APScheduler runs within the Flask application process, eliminating the need for separate worker processes | |
| 2. **No External Dependencies**: Unlike Celery which requires Redis or RabbitMQ, APScheduler uses in-memory storage by default | |
| 3. **Easier Deployment**: Simplifies deployment since there are fewer components to manage | |
| 4. **Reduced Resource Usage**: Uses less memory and CPU compared to running separate Celery processes | |
| ## Changes Made | |
| ### 1. Removed Dependencies | |
| - Removed `celery` and `redis` from `requirements.txt` | |
| - Kept `apscheduler` as the scheduling library | |
| ### 2. New Scheduler Service | |
| - Created `backend/scheduler/apscheduler_service.py` to handle all scheduling tasks | |
| - Implemented content generation and post publishing as APScheduler jobs | |
| ### 3. Updated Flask Application | |
| - Modified `backend/app.py` to initialize APScheduler instead of Celery | |
| - Added scheduler initialization when `SCHEDULER_ENABLED` is True | |
| ### 4. Updated API Endpoints | |
| - Modified `backend/api/schedules.py` to trigger APScheduler updates instead of Celery tasks | |
| - Removed all references to Celery task IDs in responses | |
| ### 5. Simplified Startup Script | |
| - Updated `start_app.py` to remove Celery component initialization | |
| - The application now starts with just Flask and APScheduler | |
| ### 6. Removed Files | |
| - Removed `backend/celery_app.py` | |
| - Removed `backend/celery_config.py` | |
| - Removed `backend/celery_tasks/` directory | |
| - Removed `backend/start_celery.py` | |
| ## Migration Steps | |
| ### 1. Update Dependencies | |
| ```bash | |
| # Update requirements | |
| pip install -r backend/requirements.txt | |
| ``` | |
| ### 2. Update Environment Variables | |
| No changes needed for basic setup. The scheduler is enabled by default. | |
| ### 3. Remove Old Components | |
| The old Celery components have been removed from the codebase. | |
| ### 4. Verify Functionality | |
| 1. Start the application: | |
| ```bash | |
| python start_app.py | |
| ``` | |
| 2. Create a test schedule via the API | |
| 3. Check the console logs to verify that schedules are being loaded and tasks are being created | |
| ## Benefits of the Migration | |
| 1. **Simpler Setup**: No need to install and configure Redis | |
| 2. **Easier Debugging**: All logs are in one place | |
| 3. **Reduced Complexity**: Fewer moving parts to manage | |
| 4. **Better Resource Usage**: Lower memory and CPU footprint | |
| 5. **Simplified Deployment**: Single process deployment | |
| ## Potential Considerations | |
| 1. **Scalability**: For high-volume applications, Celery with multiple workers might be more appropriate | |
| 2. **Persistence**: APScheduler uses in-memory storage by default, which means schedules are lost on application restart (this is mitigated by reloading from the database every 5 minutes) | |
| 3. **Task Isolation**: All tasks run in the same process, so a long-running task could block others | |
| ## Support | |
| If you encounter issues during migration: | |
| 1. Check the application logs for error messages | |
| 2. Verify that all dependencies are correctly installed | |
| 3. Ensure the Supabase connection is working | |
| 4. Test creating and deleting schedules via the API |