Gravelamps
gravelamps.core.io¶
Functions dealing with reading or writing files.
- gravelamps.core.io.create_job_string(outdir: Path, label: str, bash_fp: Path, slurm_settings: dict, dependencies_below: int = None) str¶
Creates a slurm job string from the settings.
- Parameters:
- outdir: pathlib.Path
Directory containing output from the run.
- label: str
Label for the job to be added.
- bash_fp: pathlib.Path
Path to the bash file for the job.
- slurm_settings: dict
User specified settings for the job.
- dependencies_below: int, optional, default=None
Job indices between 0 and this number will be added as dependencies to the job.
- Returns:
- job_string: str
String of attributes for the job concluding with the bash file
- gravelamps.core.io.file_index(data_dir: Path, pattern: str, suffix: str) Path¶
Yields file of the form
pattern-idx````suffixfrom thedata_dirso long as the file exists.- Parameters:
- data_dir: pathlib.Path
Directory containing patterned files.
- pattern: str
Static component of patterned file name.
- suffix: str
Suffix of file names.
- Yields:
- fp: pathlib.Path
File with increasing increment.
- gravelamps.core.io.identify_full_pattern(data_dir: Path, pattern: str) tuple[str, str]¶
Identifies the pattern of files based on a partial pattern. Cannot disambiguate.
- Parameters:
- data_dir: pathlib.Path
Folder containing patterned files.
- pattern: str
Partial pattern used to name files.
- Returns:
- full_pattern: str
The full pattern of the files containing the partial pattern.
- suffix: str
Returns the suffix for the pattern.
- Raises:
- PatternError
If the pattern could fatch a number of file patterns. If the pattern returned no matches.
- gravelamps.core.io.make_combination_submit_file(config: ConfigParser, pattern: str) tuple[dict, Path]¶
Creates an HTCondor submission file for gravelamps_combine_lens.
- Parameters:
- configConfigParser
INI configuration values.
- patternstr
Pattern to be passed to gravelamps_combine_lens
- Returns:
- condor_settings: dict
HTCondor settings for the job
- submit_fp: pathlib.Path
Path to the saved submit file.
- gravelamps.core.io.make_generation_dag(config: ConfigParser) Path¶
Creates an HTCondor DAG file for lens generation.
- Parameters:
- config: ConfigParser
INI configuration values.
- Returns:
- dag_fp: pathlib.Path
Path to generated DAG file.
- gravelamps.core.io.make_generation_submit_file(config: ConfigParser, ini_fp: str) tuple[dict, Path]¶
Creates an HTCondor submission file for gravelamps_generate_lens, returning the settings and the path
- Parameters:
- config: ConfigParser
INI configuration values.
- ini_fp: str
Path of INI file.
- Returns:
- condor_settings: dict
HTCondor settings for the job
- submit_fp: pathlib.Path
Path to the saved submit file.
- gravelamps.core.io.make_slurm_combination_bash_file(config: ConfigParser, pattern: str) Path¶
Generates a slurm-compatible bash file for gravelamps_combine_lens.
- Parameters:
- config: ConfigParser
INI configuration values.
- pattern: str
Pattern to be passed to gravelamps_combine_lens.
- Returns:
- bash_fp: pathlib.Path
Path to the saved submit file.
- gravelamps.core.io.make_slurm_generation_bash_file(config: ConfigParser, ini_fp) Path¶
Generates a slurm-compatible bash file for gravelamps_generate_lens.
- Parameters:
- config: ConfigParser
INI configuration values.
- ini_fp: str
Path of INI file.
- Returns:
- bash_fp: pathlib.Path
Path to the saved bash file
- gravelamps.core.io.make_slurm_generation_bash_files(config: ConfigParser, ini_fp: str) list[Path] | Path¶
Generates the bash file(s) that are needed to run the jobs through slurm.
- Parameters:
- config: ConfigParser
INI Configuration values.
- ini_fp:
Path to INI file.
- Returns:
- bash_fp: list[pathlib.Path] | pathlib.Path
Path to the submission file(s) for the generation (and combination) job(s).
- gravelamps.core.io.make_slurm_generation_batch_file(config: ConfigParser, bash_fp: Path | list[Path]) Path¶
Creates the overall batch file for the generation (and combination) job(s) and ensures that job relations are correctly set up.
- Parameters:
- config: ConfigParser
INI Configuration values.
- bash_fp: pathlib.Path | list[pathlib.Path]
File(s) containing generation (and combination) job(s).
- Returns:
- submit_fp: pathlib.Path
Location of batch file.
- gravelamps.core.io.write_bilby_pipe_ini(bilby_pipe_config: dict) Path¶
Writes a bilby_pipe INI file from the configuration. Will place the file with the name bilbypipe.ini in the bilby_pipe output directory—defaulting to the CWD.
- Parameters:
- bilby_pipe_config: dict
bilby_pipe configuration.
- Returns:
- fp: pathlib.Path
Path to INI file.
- gravelamps.core.io.write_condor_file(settings: dict, fp: Path, queue: int = 1) None¶
Write a settings dictionary to an HTCondor submit file.
- Parameters:
- settings: dict
HTCondor job configuration
- fp: pathlib.Path
Path to save file to
- queue: int, optional, default=1
Number of jobs to queue
- gravelamps.core.io.write_inference_dag(outdir: Path, config: ConfigParser) Path¶
Makes a DAG containing all necessary jobs for the inference program.
- Parameters:
- outdir: pathlib.Path
Path to place output in.
- config: ConfigParser
INI configuration values.
- Returns:
- dag_fp: pathlib.Path
Path to the generated DAG file.
- Raises:
- FileNotFoundError
If the output directory doesn’t exist.
- gravelamps.core.io.write_outermost_slurm_batch_file(config: ConfigParser, outdir: Path) Path¶
Write the outermost batch file for gravelamps_inference.
This will find the generation batch file(s) and the bilby analysis batch file and create an outermost file that runs these jobs.
- Parameters:
- config: ConfigParser
INI configuration values.
- outdir: pathlib.Path
Directory to output results to.
- Returns:
- batch_fp: pathlib.Path
Path of the newly written outermost batch file.