oioioi.sinolpack.package

Module Contents

Classes

SinolPackage
SinolPackageCreator	Responsible for packing SinolPackages.
SinolPackageBackend	Backend that use

Functions

_stringify_keys(dictionary)
_determine_encoding(title, file)
_decode(title, file)
_make_filename_in_job_dir(env, base_name)
_remove_from_zip(zipfname, *filenames)	Removes files from zip file by creating new zip file with all

Attributes

logger
DEFAULT_TIME_LIMIT
DEFAULT_MEMORY_LIMIT
TASK_PRIORITY
C_EXTRA_ARGS
PAS_EXTRA_ARGS

oioioi.sinolpack.package.logger

oioioi.sinolpack.package.DEFAULT_TIME_LIMIT = 10000

oioioi.sinolpack.package.DEFAULT_MEMORY_LIMIT = 66000

oioioi.sinolpack.package.TASK_PRIORITY = 500

oioioi.sinolpack.package.C_EXTRA_ARGS = ['-Wall', '-Wno-unused-result', '-Werror']

oioioi.sinolpack.package.PAS_EXTRA_ARGS = ['-Ci', '-Cr', '-Co', '-gl']

oioioi.sinolpack.package._stringify_keys(dictionary)

oioioi.sinolpack.package._determine_encoding(title, file)

oioioi.sinolpack.package._decode(title, file)

oioioi.sinolpack.package._make_filename_in_job_dir(env, base_name)

oioioi.sinolpack.package._remove_from_zip(zipfname, *filenames)

Removes files from zip file by creating new zip file with all the files except the files to remove. Then the old file is removed. It has to be done like this because zipfile module doesn’t implement function to delete file.

class oioioi.sinolpack.package.SinolPackage(path, original_filename=None)

Bases: object

controller_name = oioioi.sinolpack.controllers.SinolProblemController

package_backend_name = oioioi.sinolpack.package.SinolPackageBackend

identify()

get_short_name()

_find_main_dir()

Looks for the directory which contains at least the in/ and out/ subdirectories. Only one such directory should be found. Otherwise None is returned.

_save_to_field(field, file)

_find_and_compile(suffix, command=None, cwd=None, log_on_failure=True, out_name=None)

_compile_using_make(command, cwd, suffix)

_compile_matching_extension(command, out_name, suffix)

_compile(filename, prog_name, ext, out_name=None)

_override_compiler(prefix, lang, compilation_job)

_run_compilation_job(ext, ft_source_name, out_name)

_ensure_compilation_success(filename, new_env)

_make_ins(re_string)

unpack(env, package)

_create_problem_or_reuse_if_exists(existing_problem)

_ensure_short_name_equality_with(existing_problem)

_create_problem_instance()

_extract_and_process_package()

_describe_processing_error()

_process_package()

_process_config_yml()

Parses the config.yml file from the package.

Extracted information is then saved for later use.

_detect_full_name()

Sets the problem’s full name from the config.yml (key title) or from the title tag in the LaTeX source file. The config.yml file takes precedence over the LaTeX source.

Example of how the title tag may look like:

itle{A problem}

_detect_full_name_translations()

Creates problem’s full name translations from the config.yml (keys matching the pattern title_[a-z]{2}, where [a-z]{2} represents two-letter language code defined in settings.py), if any such key is given.

_detect_library()

Finds if the problem has a library defined in config.yml.

Tries to read a library name (filename library should be given during compilation) from the config.yml (key library).

If there is no such key, assumes that a library is not needed.

_process_extra_files()

Looks for extra compilation files specified in config.yml.

_find_and_save_files(files)

Saves files in the database.

Parameters

files – List of expected files.

Returns

List of files that were not found.

_extract_makefiles()

_save_prog_dir()

Creates an archive to store compiled programs.

_process_statements()

Creates a problem statement from html or pdf source.

If USE_SINOLPACK_MAKEFILES is set to True in the OIOIOI settings, the pdf file will be compiled from a LaTeX source.

_force_index_encoding(htmlzipfile)

Ensures index.html file is utf-8 encoded, if cannot apply this encoding raise ProblemPackageError.

_compile_latex_docs(docdir)

_generate_tests(total_score_if_auto=100)

Generates tests for the problem.

First, time and memory limits are obtained from config.yml. Then, the judge system attempts to obtain a generic memory limit from the problem statement.

Next, test instances are created, using input files (*.in) provided with the package and ingen (the input file generator) where applicable (ingen-generated tests may replace *.in files contained in the package if their respective names coincide).

If USE_SINOLPACK_MAKEFILES is set to True in the OIOIOI settings, output files (*.out) will also be generated at this point, based on the model solution’s output (on condition that its source code is included within the package).

The next step is veryfing whether the sum of time limits over all tests does not exceed the maximum defined by the OIOIOI installation’s owner.

If an input verifier is provided, it will then assert that all *.in files generated are valid, and abort the upload in case of failure.

Here the *.out files will be generated if USE_SINOLPACK_MAKEFILES is set to False, based on the model solution’s output (on condition that its source code is included within the package).

In the end, it is asserted that all tests have been correctly constructed, non-created tests are removed from the database and test scores are assigned to tests and testgroups based on the configuration from config.yml or set to default value if not specified.

_detect_statement_memory_limit()

Returns the memory limit in the problem statement, converted to KiB or None.

_create_instances_for_tests()

Iterate through available test inputs. :return: Triple (created tests instances,

outs that have to be generated, score groups (determined by test names))

_verify_time_limits(time_limit_sum)

Checks whether the sum of test time limits does not exceed the allowed maximum.

Raises

ProblemPackageError

if sum of tests time limits exceeds the maximum defined in the settings.py file.

_verify_inputs(tests)

Checks if inwer exits with code 0 on all tests.

Raises

ProblemPackageError

otherwise.

_generate_test_outputs(tests, outs_to_make)

_validate_tests(created_tests)

Checks if all tests have output files and that all tests have been successfully created.

Raises

ProblemPackageError

_delete_non_existing_tests(created_tests)

_process_test(test, order, names_re, indir, outdir, collected_ins, scored_groups, outs_to_make)

Responsible for saving test in and out files, setting test limits, assigning test kinds and groups.

Parameters

• test – Test name.

• order – Test number.

• names_re – Compiled regex to match test details from name. Should extract basename, test name, group number and test type.

• indir – Directory with tests inputs.

• outdir – Directory with tests outputs.

• collected_ins – List of inputs that were generated, not taken from archive as a file.

• scored_groups – Accumulator for score groups.

• outs_to_make – Accumulator for name of output files to be generated by model solution.

Returns

Test instance or None if name couldn’t be matched.

_get_memory_limit(created, name, group)

If we find the memory limit specified anywhere in the package: either in the config.yml or in the problem statement then we overwrite potential manual changes.

(In the future we should disallow editing memory limits if they were taken from the package).

The memory limit is more important the more specific it is. In particular, the global memory limit is less important than the memory limit for a test group, while the memory limit for particular test is the most imporant. :return: Memory limit found in config or statement,

None otherwise.

_get_time_limit(created, name, group)

If we find the time limit specified anywhere in in the config.yml then we overwrite potential manual changes.

The time limit is more important the more specific it is. In particular, the global time limit is less important than the time limit for a test group, while the time limit for particular test is the most imporant. :return: Time limit found in config, None otherwise.

_make_outs(outs_to_make)

Compiles the model solution and executes it in order to generate test outputs.

Returns

Result from workers.

_check_scores_from_config(scored_groups, config_scores)

Called if config.yml specifies scores for any tests. Makes sure that all scored tests are present in config.yml and that nothing else is there.

_compute_scores_automatically(scored_groups, total_score)

If there are no testscores specified config.yml, all groups get equal score, except few last groups that are given +1 to compensate rounding error and match the total sum of total_score.

_assign_scores(scored_groups, total_score_if_auto)

Checks if there’s a scores entry in config and sets scores according to that or assigns them automatically otherwise.

_process_language_override()

Checks if there’s a override_limits entry in config and for existing tests, add additional limits overrides. Time limits are validated the same way it’s validated in default package.

_prepare_overrides(lang)

Prepares overrides for specified language, initially setting to default limits.

_set_memory_limit_overrides(lang, rules)

Sets memory limits overrides for specific language.

_set_time_limit_overrides(lang, rules)

Sets time limits overrides for specific language.

_process_checkers()

Compiles an output checker and saves its binary.

_find_checker_exec()

_process_model_solutions()

Saves model solutions to the database.

_get_model_solutions_sources()

Returns

Sources as tuples (kind_of_solution, filename,

full path to file).

_process_attachments()

Removes previously added attachments for the problem, and saves new ones from the attachment directory.

_save_original_package()

Save instance of package that would be reused by other instances of this problem.

_html_disallowed()

class oioioi.sinolpack.package.SinolPackageCreator(problem)

Bases: object

Responsible for packing SinolPackages.

pack()

Returns

Archive from original package if such file exists,

otherwise new archive with test, statements and model solutions.

_create_basic_archive()

Produce the most basic output: tests, statements, model solutions.

_pack_statement()

_pack_django_file(django_file, arcname)

Packs file represented by :class:~`oioioi.filetracker.fields.FileField`

_pack_tests()

_pack_model_solutions()

class oioioi.sinolpack.package.SinolPackageBackend

Bases: oioioi.problems.package.ProblemPackageBackend

Backend that use SinolPackage to unpack and SinolPackageCreator to pack sinol packages.

description

package_class

identify(path, original_filename=None)

Checks if the backend is suitable for processing the specified problem package.

Parameters

• path – a path to the processed problem package

• original_filename – the name of the package specified by the

uploading user.

Returns True if the backend can handle the specified problem package file.

get_short_name(path, original_filename=None)

Returns the problem’s short name.

Parameters

• path – a path to the processed problem package

• original_filename – the name of the package specified by the

uploading user.

unpack(env)

Processes a package, creating a new Problem or updating an existing one.

This function will be called either from unpackmgr_job() (Celery task) or from simple_unpack() (e.g. when a problem is added from a command line).

Used env keys:

package_id: an id of the ProblemPackage instance with the package file to unpack.

Produced env keys:

problem_id: an id of the Problem instance representing the created or modified problem.

pack(problem)

Creates a package from problem, returns a django.http.HttpResponse instance.

Should raise NotImplementedError if creating packages is not supported.