oioioi.evalmgr.tasks

Module Contents

Functions

_placeholder(environ, **kwargs)
_find_placeholder(recipe, name)
find_recipe_entry(recipe, name)
recipe_placeholder(name)
create_environ()	Creates new environ, filling most basic fields. This fields can't be
transfer_job(environ, transfer_func, restore_func[, ...])	Transfers job to external evaluation system using given transfer
add_after_placeholder(environ, name, entry)
add_before_placeholder(environ, name, entry)
extend_after_placeholder(environ, name, entries)
extend_before_placeholder(environ, name, entries)
add_after_recipe_entry(environ, name, entry)
add_before_recipe_entry(environ, name, entry)
replace_recipe_entry(environ, name, new_entry)
_run_phase(env, phase[, extra_kwargs])
_resume_job(environ)	Restores saved environ, returns environ or None, when a matching
_job_finished(environ)
_transfer_job(environ, transfer_func, transfer_kwargs)
_mark_job_state(environ, state)
_run_error_handlers(env, exc_info)
delay_environ(environ, **evalmgr_extra_args)	Inserts environ into evalmgr queue with marking it as queued, resuming
evalmgr_job(env)	Takes environment and evaluates it according to its recipe.

Attributes

loaded_controllers

oioioi.evalmgr.tasks.loaded_controllers = False

oioioi.evalmgr.tasks._placeholder(environ, **kwargs)

oioioi.evalmgr.tasks._find_placeholder(recipe, name)

oioioi.evalmgr.tasks.find_recipe_entry(recipe, name)

oioioi.evalmgr.tasks.recipe_placeholder(name)

oioioi.evalmgr.tasks.create_environ()

Creates new environ, filling most basic fields. This fields can’t be overwritten (they can be extended, etc): - job_id - error_handlers

oioioi.evalmgr.tasks.transfer_job(environ, transfer_func, restore_func, transfer_kwargs=None)

Transfers job to external evaluation system using given transfer function. Transfer function will be called with environ as first positional argument and **transfer_kwargs as keyword arguments.

Environ is saved by evalmgr right before transfer, so the transfer function may send out only the information needed for the external evaluation system.

The transfer_func isn’t called from a database transaction, and thus shouldn’t use database (unless it makes a transaction on its own).

To resume a job, delay_environ must be called with the resulting environment as the argument, which must contain saved_environ_id. This field is added to the environ right before transfer_func is called (so it isn’t included in environ saved in database, but is in one handled as argument to transfer_func).

When job is resumed, evalmgr searches for a matching SavedEnviron. If it finds any, environ is updated with the external evaluation results (the delay_environ’s argument) like:

environ = restore_func(saved_environ, results_environ).

If it doesn’t find any, job is considered already resumed and is ignored.

For details see evalmgr_job and delay_environ.

oioioi.evalmgr.tasks.add_after_placeholder(environ, name, entry)

oioioi.evalmgr.tasks.add_before_placeholder(environ, name, entry)

oioioi.evalmgr.tasks.extend_after_placeholder(environ, name, entries)

oioioi.evalmgr.tasks.extend_before_placeholder(environ, name, entries)

oioioi.evalmgr.tasks.add_after_recipe_entry(environ, name, entry)

oioioi.evalmgr.tasks.add_before_recipe_entry(environ, name, entry)

oioioi.evalmgr.tasks.replace_recipe_entry(environ, name, new_entry)

oioioi.evalmgr.tasks._run_phase(env, phase, extra_kwargs=None)

oioioi.evalmgr.tasks._resume_job(environ)

Restores saved environ, returns environ or None, when a matching SavedEnviron wasn’t found.

oioioi.evalmgr.tasks._job_finished(environ)

oioioi.evalmgr.tasks._transfer_job(environ, transfer_func, transfer_kwargs)

oioioi.evalmgr.tasks._mark_job_state(environ, state)

oioioi.evalmgr.tasks._run_error_handlers(env, exc_info)

oioioi.evalmgr.tasks.delay_environ(environ, **evalmgr_extra_args)

Inserts environ into evalmgr queue with marking it as queued, resuming it if it should be. Returns associated async result, or None when job was already resumed before (or was cancelled).

Requires to be called from transaction.

oioioi.evalmgr.tasks.evalmgr_job(env)

Takes environment and evaluates it according to its recipe.

To queue environment one shouldn’t call evalmgr_job.apply_async or evalmgr_job.delay directly. Use delay_environ instead.

It needs some env elements to be set:

env['recipe'] should be a list of tuples

(phaseName, handler_fun [, kwargs])

handler_fun should satisfy this definition:

def handler_fun(env, \*\*kwargs):

kwargs will be passed to handler_fun

current environment would be passed as env

It’s guaranteed that handler functions would be run in order they’re listed and that environment modified by previous function will be passed to next function unchanged.

Before running the job, its unique identifier is generated and saved in env['job_id'].

A job may be transferred to an external evaluation system using transfer_job. A recipe handler which want to transfer job should return environ processed by transfer_job, like:

def prepare_transfer_handler(environ, **kwargs):
  do_some_important_stuff(environ, **kwargs)
  return transfer_job(environ,
                      'some.module.transfer_function',
                      'functions.that.merges.saved.env.and.result')

If during any of the phases an exception other than Ignore is thrown, and env['error_handlers'] is present (it should be in the same format as recipe), functions listed there are called with additional exc_info argument, which is the sys.exc_info() triple. If any exceptions are thrown there, they are reported to the logs and ignored.

If a celery.exceptions.Ignore is thrown, then execution of environment is stopped. One who does it is responsible for handling corresponding QueuedJob object.

Returns environment (a processed copy of given environment).