Flask with create_app, SQLAlchemy and Celery -
i'm struggling proper setup flask, sqlalchemy , celery. have searched extensively , tried different approaches, nothing seems work. either missed application context or can't run workers or there other problems. structure general can build bigger application.
i'm using: flask 0.10.1, sqlalchemy 1.0, celery 3.1.13, current setup following:
app/__init__.py
#empty app/config.py
import os basedir = os.path.abspath(os.path.dirname(__file__)) class config: @staticmethod def init_app(app): pass class localconfig(config): debug = true sqlalchemy_database_uri = r"sqlite:///" + os.path.join(basedir, "data-dev.sqlite") celery_broker_url = 'amqp://guest:guest@localhost:5672//' config = { "local": localconfig} app/exstensions.py
from flask.ext.sqlalchemy import sqlalchemy celery import celery db = sqlalchemy() celery = celery() app/factory.py
from extensions import db, celery flask import flask flask import g config import config def create_before_request(app): def before_request(): g.db = db return before_request def create_app(config_name): app = flask(__name__) app.config.from_object(config[config_name]) db.init_app(app) celery.config_from_object(config) # register blueprints # add before request handler app.before_request(create_before_request(app)) return app app/manage.py
from factory import create_app app = create_app("local") flask import render_template flask import request @app.route('/test', methods=['post']) def task_simple(): import tasks tasks.do_some_stuff.delay() return "" if __name__ == "__main__": app.run() app/models.py
from extensions import db class user(db.model): __tablename__ = "user" id = db.column(db.integer, primary_key=true) username = db.column(db.string(128), unique=true, nullable=false) app/tasks.py
from extensions import celery celery.signals import task_prerun flask import g, current_app @task_prerun.connect def close_session(*args, **kwargs): current_app.app_context(): # use g.db print g @celery.task() def do_some_stuff(): current_app.app_context(): # use g.db print g in folder app:
- starting development webserver with:
python.exe manage.py - starting workers with:
celery.exe worker -a tasks
i import error doesn't make sense me. should structure application differently? @ end think want quite basic setup, e.g. using flask factory pattern, able use flask-sqlalchmey extension , have worker needs access database.
any highly appreciated.
traceback executed when starting celery worker.
traceback (most recent call last): file "[path]\scripts\celery-script.py", line 9, in <module> load_entry_point('celery==3.1.13', 'console_scripts', 'celery')() file "[path]\lib\site-packages\celery\__main__.py", line 30, in main main() file "[path]\lib\site-packages\celery\bin\celery.py", line 81, in main cmd.execute_from_commandline(argv) file "[path]\lib\site-packages\celery\bin\celery.py", line 769, in execute_from_commandline super(celerycommand, self).execute_from_commandline(argv))) file "[path]\lib\site-packages\celery\bin\base.py", line 305, in execute_from_commandline argv = self.setup_app_from_commandline(argv) file "[path]\lib\site-packages\celery\bin\base.py", line 473, in setup_app_from_commandline user_preload = tuple(self.app.user_options['preload'] or ()) attributeerror: 'flask' object has no attribute 'user_options' update change code according suggestion in comment. worker starts when test request http://127.0.0.1:5000/test. following traceback:
traceback (most recent call last): file "[path]\lib\site-packages\celery\app\trace.py", line 230, in trace_task args=args, kwargs=kwargs) file "[path]\lib\site-packages\celery\utils\dispatch\signal.py", line 166, in send response = receiver(signal=self, sender=sender, \**named) file "[path]\app\stackoverflow\tasks.py", line 7, in close_session current_app.app_context(): file "[path]\lib\site-packages\werkzeug\local.py", line 338, in __getattr__ return getattr(self._get_current_object(), name) file "[path]\lib\site-packages\werkzeug\local.py", line 297, in _get_current_object return self.__local() file "[path]\lib\site-packages\flask\globals.py", line 34, in _find_app raise runtimeerror('working outside of application context') runtimeerror: working outside of application context exc, exc_info.traceback))) update based on comments marteen, changed code. current working version under: https://gist.github.com/anonymous/fa47834db2f4f3b8b257. further improvements or suggestions welcome.
i off current_app advice.
your celery object needs access application context. found information online creating celery object factory function. example below tested without message broker.
#factory.py celery import celery config import config def create_celery_app(app=none): app = app or create_app(config) celery = celery(__name__, broker=app.config['celery_broker_url']) celery.conf.update(app.config) taskbase = celery.task class contexttask(taskbase): abstract = true def __call__(self, *args, **kwargs): app.app_context(): return taskbase.__call__(self, *args, **kwargs) celery.task = contexttask return celery and in tasks.py:
#tasks.py factory import create_celery_app celery.signals import task_prerun flask import g celery = create_celery_app() @task_prerun.connect def celery_prerun(*args, **kwargs): #print g celery.app.app_context(): # # use g.db print g @celery.task() def do_some_stuff(): celery.app.app_context(): # use g.db g.user = "test" print g.user some links:
flask pattern creating celery instance factory function
application using both application factory , celery
Comments
Post a Comment