# sampling-utils - Resampling Utilities¶

## Overview¶

New in version 0.3.1.

### Resampling Utilities¶

#### sample command¶

This command samples from a Fasta or FastQ file, based on a probability defined by the user (0.001 or 1 / 1000 by default, -r parameter), for a maximum number of sequences (100,000 by default, -x parameter). By default 1 sample is extracted, but as many as desired can be taken, by using the -n parameter.

The sequence file in input can be either be passed to the standard input or as last parameter on the command line. By defult a Fasta is expected, unless the -q parameter is passed.

The -p parameter specifies the prefix to be used, and if the output files can be gzipped using the -z parameter.

#### sample_stream command¶

It works in the same way as sample, however the file is sampled only once and the output is the stdout by default. This can be convenient if streams are a preferred way to sample the file.

#### sync command¶

Used to keep in sync forward and reverse read files in paired-end FASTQ. The scenario is that the sample command was used to resample a FASTQ file, usually the forward, but we need the reverse as well. In this case, the resampled file, called master is passed to the -m option and the input file is the file that is to be synced (reverse). The input file is scanned until the same header is found in the master file and when that happens, the sequence is written. The next sequence is then read from the master file and the process is repeated until all sequence in the master file are found in the input file. This implies having the 2 files sorted in the same way, which is what the sample command does.

Note

the old casava format is not supported by this command at the moment, as it’s unusual to find it in SRA or other repositories as well.

#### rand_seq command¶

Generate random FastA/Q sequences, allowing the specification of GC content and number of sequences being coding or random. If the output format chosen is FastQ, qualities are generated using a decreasing model with added noise. A constant model can be specified instead with a switch. Parameters such GC, length and the type of model can be infered by passing a FastA/Q file, with the quality model fit using a LOWESS (using mgkit.utils.sequence.extrapolate_model()). The noise in that case is model as the a normal distribution fitted from the qualities along the sequence deviating from the fitted LOWSS and scaled back by half to avoid too drastic changes in the qualities. Also the qualities are clipped at 40 to avoid compatibility problems with FastQ readers. If inferred, the model can be saved (as a pickle file) and loaded back for analysis

#### Changes¶

Changed in version 0.3.4: using click instead of argparse. Now *rand_seq can save and reload models

## Options¶

### sampling-utils¶

Main function

sampling-utils [OPTIONS] COMMAND [ARGS]...


Options

--version

Show the version and exit.

--cite

#### rand_seq¶

Generates random FastA/Q sequences

sampling-utils rand_seq [OPTIONS] [OUTPUT_FILE]


Options

-v, --verbose
-n, --num-seqs <num_seqs>

Number of sequences to generate [default: 1000]

-gc, --gc-content <gc_content>

GC content (defaults to .5 out of 1) [default: 0.5]

-i, --infer-params <infer_params>

Infer parameters GC content and Quality model from file

-r, --coding-prop <coding_prop>

Proportion of coding sequences [default: 0.0]

-l, --length <length>

Sequence length [default: 150]

-d, --const-model

Use a model with constant qualities + noise

-x, --dist-loc <dist_loc>

Use as the starting point quality [default: 30.0]

-q, --fastq

The output file is a FastQ file

-m, --save-model <save_model>

Save inferred qualities model to a pickle file

-a, --read-model <read_model>

Load qualities model from a pickle file

Arguments

OUTPUT_FILE

Optional argument

#### sample¶

Sample a FastA/Q multiple times

sampling-utils sample [OPTIONS] [INPUT_FILE]


Options

-v, --verbose
-p, --prefix <prefix>

Prefix for the file name(s) in output [default: sample]

-n, --number <number>

Number of samples to take [default: 1]

-r, --prob <prob>

Probability of picking a sequence [default: 0.001]

-x, --max-seq <max_seq>

Maximum number of sequences [default: 100000]

-q, --fastq

The input file is a fastq file

-z, --gzip

gzip output files

Arguments

INPUT_FILE

Optional argument

#### sample_stream¶

Samples a FastA/Q one time, alternative to sample if multiple sampling is not needed

sampling-utils sample_stream [OPTIONS] [INPUT_FILE] [OUTPUT_FILE]


Options

-v, --verbose
-r, --prob <prob>

Probability of picking a sequence

-x, --max-seq <max_seq>

Maximum number of sequences

-q, --fastq

The input file is a fastq file

Arguments

INPUT_FILE

Optional argument

OUTPUT_FILE

Optional argument

#### sync¶

Syncs a FastQ file generated with sample with the original pair of files.

sampling-utils sync [OPTIONS] [INPUT_FILE] [OUTPUT_FILE]


Options

-v, --verbose
-m, --master-file <master_file>

Resampled FastQ file that is out of sync with the original pair [required]

Arguments

INPUT_FILE

Optional argument

OUTPUT_FILE

Optional argument