oioioi.evalmgr.tasks
¶
Module Contents¶
Functions¶
|
|
|
|
|
|
|
|
Creates new environ, filling most basic fields. This fields can't be |
|
|
Transfers job to external evaluation system using given transfer |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Restores saved environ, returns environ or None, when a matching |
|
|
|
|
|
|
|
|
|
Inserts environ into evalmgr queue with marking it as queued, resuming |
|
Takes environment and evaluates it according to its recipe. |
Attributes¶
- oioioi.evalmgr.tasks.create_environ()[source]¶
Creates new environ, filling most basic fields. This fields can’t be overwritten (they can be extended, etc): - job_id - error_handlers
- oioioi.evalmgr.tasks.transfer_job(environ, transfer_func, restore_func, transfer_kwargs=None)[source]¶
Transfers job to external evaluation system using given transfer function. Transfer function will be called with environ as first positional argument and **transfer_kwargs as keyword arguments.
Environ is saved by evalmgr right before transfer, so the transfer function may send out only the information needed for the external evaluation system.
The transfer_func isn’t called from a database transaction, and thus shouldn’t use database (unless it makes a transaction on its own).
To resume a job,
delay_environ
must be called with the resulting environment as the argument, which must containsaved_environ_id
. This field is added to the environ right beforetransfer_func
is called (so it isn’t included in environ saved in database, but is in one handled as argument totransfer_func
).When job is resumed, evalmgr searches for a matching SavedEnviron. If it finds any, environ is updated with the external evaluation results (the
delay_environ
’s argument) like:environ = restore_func(saved_environ, results_environ).
If it doesn’t find any, job is considered already resumed and is ignored.
For details see
evalmgr_job
anddelay_environ
.
- oioioi.evalmgr.tasks._resume_job(environ)[source]¶
Restores saved environ, returns environ or None, when a matching SavedEnviron wasn’t found.
- oioioi.evalmgr.tasks.delay_environ(environ, **evalmgr_extra_args)[source]¶
Inserts environ into evalmgr queue with marking it as queued, resuming it if it should be. Returns associated async result, or None when job was already resumed before (or was cancelled).
Requires to be called from transaction.
- oioioi.evalmgr.tasks.evalmgr_job(env)[source]¶
Takes environment and evaluates it according to its recipe.
To queue environment one shouldn’t call
evalmgr_job.apply_async
orevalmgr_job.delay
directly. Usedelay_environ
instead.It needs some env elements to be set:
env['recipe']
should be a list of tuples(
phaseName
,handler_fun
[,kwargs
])- handler_fun should satisfy this definition:
def handler_fun(
env
,\*\*kwargs
):
kwargs will be passed to
handler_fun
current environment would be passed as
env
It’s guaranteed that handler functions would be run in order they’re listed and that environment modified by previous function will be passed to next function unchanged.
Before running the job, its unique identifier is generated and saved in
env['job_id']
.A job may be transferred to an external evaluation system using
transfer_job
. A recipe handler which want to transfer job should return environ processed bytransfer_job
, like:def prepare_transfer_handler(environ, **kwargs): do_some_important_stuff(environ, **kwargs) return transfer_job(environ, 'some.module.transfer_function', 'functions.that.merges.saved.env.and.result')
If during any of the phases an exception other than Ignore is thrown, and
env['error_handlers']
is present (it should be in the same format as recipe), functions listed there are called with additionalexc_info
argument, which is thesys.exc_info()
triple. If any exceptions are thrown there, they are reported to the logs and ignored.If a celery.exceptions.Ignore is thrown, then execution of environment is stopped. One who does it is responsible for handling corresponding QueuedJob object.
Returns environment (a processed copy of given environment).