1 1.1 christos 2 1.1 christos #------------------------------------------------------------------------------ 3 1.1.1.2 christos # $File: bioinformatics,v 1.5 2019/04/19 00:42:27 christos Exp $ 4 1.1 christos # bioinfomatics: file(1) magic for Bioinfomatics file formats 5 1.1 christos 6 1.1 christos ############################################################################### 7 1.1 christos # BGZF (Blocked GNU Zip Format) - gzip compatible, but also indexable 8 1.1 christos # used by SAMtools bgzip/tabix (http://samtools.sourceforge.net/tabix.shtml) 9 1.1 christos ############################################################################### 10 1.1 christos 0 string \037\213 11 1.1 christos >3 byte &0x04 12 1.1 christos >>12 string BC 13 1.1 christos >>>14 leshort &0x02 Blocked GNU Zip Format (BGZF; gzip compatible) 14 1.1 christos >>>>16 leshort x \b, block length %d 15 1.1 christos !:mime application/x-gzip 16 1.1 christos 17 1.1 christos 18 1.1 christos ############################################################################### 19 1.1 christos # Tabix index file 20 1.1 christos # used by SAMtools bgzip/tabix (http://samtools.sourceforge.net/tabix.shtml) 21 1.1 christos ############################################################################### 22 1.1 christos 0 string TBI\1 SAMtools TBI (Tabix index format) 23 1.1 christos >0x04 lelong =1 \b, with %d reference sequence 24 1.1 christos >0x04 lelong >1 \b, with %d reference sequences 25 1.1 christos >0x08 lelong &0x10000 \b, using half-closed-half-open coordinates (BED style) 26 1.1 christos >0x08 lelong ^0x10000 27 1.1 christos >>0x08 lelong =0 \b, using closed and one based coordinates (GFF style) 28 1.1 christos >>0x08 lelong =1 \b, using SAM format 29 1.1 christos >>0x08 lelong =2 \b, using VCF format 30 1.1 christos >0x0c lelong x \b, sequence name column: %d 31 1.1 christos >0x10 lelong x \b, region start column: %d 32 1.1 christos >0x08 lelong =0 33 1.1 christos >>0x14 lelong x \b, region end column: %d 34 1.1 christos >0x18 byte x \b, comment character: %c 35 1.1 christos >0x1c lelong x \b, skip line count: %d 36 1.1 christos 37 1.1 christos 38 1.1 christos ############################################################################### 39 1.1 christos # BAM (Binary Sequence Alignment/Map format) 40 1.1 christos # used by SAMtools (http://samtools.sourceforge.net/SAM1.pdf) 41 1.1 christos # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it 42 1.1 christos ############################################################################### 43 1.1 christos 0 string BAM\1 SAMtools BAM (Binary Sequence Alignment/Map) 44 1.1 christos >0x04 lelong >0 45 1.1 christos >>&0x00 regex =^[@]HD\t.*VN: \b, with SAM header 46 1.1 christos >>>&0 regex =[0-9.]+ \b version %s 47 1.1 christos >>&(0x04) lelong >0 \b, with %d reference sequences 48 1.1 christos 49 1.1 christos 50 1.1 christos ############################################################################### 51 1.1 christos # BAI (BAM indexing format) 52 1.1 christos # used by SAMtools (http://samtools.sourceforge.net/SAM1.pdf) 53 1.1 christos ############################################################################### 54 1.1 christos 0 string BAI\1 SAMtools BAI (BAM indexing format) 55 1.1 christos >0x04 lelong >0 \b, with %d reference sequences 56 1.1 christos 57 1.1 christos 58 1.1 christos ############################################################################### 59 1.1 christos # CRAM (Binary Sequence Alignment/Map format) 60 1.1 christos ############################################################################### 61 1.1 christos 0 string CRAM CRAM 62 1.1 christos >0x04 byte >-1 version %d. 63 1.1 christos >0x05 byte >-1 \b%d 64 1.1 christos >0x06 string >\0 (identified as %s) 65 1.1 christos 66 1.1 christos 67 1.1 christos ############################################################################### 68 1.1 christos # BCF (Binary Call Format), version 1 69 1.1 christos # used by SAMtools & VCFtools (http://vcftools.sourceforge.net/bcf.pdf) 70 1.1 christos # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it 71 1.1 christos ############################################################################### 72 1.1 christos 0 string BCF\4 73 1.1 christos # length of seqnm data in bytes is positive 74 1.1 christos >&0x00 lelong >0 75 1.1 christos # length of smpl data in bytes is positive 76 1.1 christos >>&(&-0x04) lelong >0 SAMtools BCF (Binary Call Format) 77 1.1 christos # length of meta in bytes 78 1.1 christos >>>&(&-0x04) lelong >0 79 1.1 christos # have meta text string 80 1.1 christos >>>>&0x00 search ##samtoolsVersion= 81 1.1 christos >>>>>&0x00 string x \b, generated by SAMtools version %s 82 1.1 christos 83 1.1 christos 84 1.1 christos ############################################################################### 85 1.1 christos # BCF (Binary Call Format), version 2.1 86 1.1.1.2 christos # used by SAMtools (https://samtools.github.io/hts-specs/BCFv2_qref.pdf) 87 1.1 christos # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it 88 1.1 christos ############################################################################### 89 1.1 christos 0 string BCF\2\1 Binary Call Format (BCF) version 2.1 90 1.1 christos # length of header text 91 1.1 christos >&0x00 lelong >0 92 1.1 christos # have header string 93 1.1 christos >>&0x00 search ##samtoolsVersion= 94 1.1 christos >>>&0x00 string x \b, generated by SAMtools version %s 95 1.1 christos 96 1.1 christos 97 1.1 christos ############################################################################### 98 1.1 christos # BCF (Binary Call Format), version 2.2 99 1.1.1.2 christos # used by SAMtools (https://samtools.github.io/hts-specs/BCFv2_qref.pdf) 100 1.1 christos # data is normally present only within compressed BGZF blocks (CDATA), so use file -z to examine it 101 1.1 christos ############################################################################### 102 1.1 christos 0 string BCF\2\2 Binary Call Format (BCF) version 2.2 103 1.1 christos # length of header text 104 1.1 christos >&0x00 lelong >0 105 1.1 christos # have header string 106 1.1 christos >>&0x00 search ##samtoolsVersion= 107 1.1 christos >>>&0x00 string x \b, generated by SAMtools version %s 108 1.1 christos 109 1.1 christos ############################################################################### 110 1.1 christos # VCF (Variant Call Format) 111 1.1 christos # used by VCFtools (http://vcftools.sourceforge.net/) 112 1.1 christos ############################################################################### 113 1.1 christos 0 search ##fileformat=VCFv Variant Call Format (VCF) 114 1.1 christos >&0 string x \b version %s 115 1.1 christos 116 1.1 christos ############################################################################### 117 1.1 christos # FASTQ 118 1.1 christos # used by MAQ (http://maq.sourceforge.net/fastq.shtml) 119 1.1 christos ############################################################################### 120 1.1 christos # XXX Broken? 121 1.1 christos # @<seqname> 122 1.1 christos #0 regex =^@[A-Za-z0-9_.:-]+\?\n 123 1.1 christos # <seq> 124 1.1 christos #>&1 regex =^[A-Za-z\n.~]++ 125 1.1 christos # +[<seqname>] 126 1.1 christos #>>&1 regex =^[A-Za-z0-9_.:-]*\?\n 127 1.1 christos # <qual> 128 1.1 christos #>>>&1 regex =^[!-~\n]+\n FASTQ 129 1.1 christos 130 1.1 christos ############################################################################### 131 1.1 christos # FASTA 132 1.1.1.2 christos # used by FASTA (https://fasta.bioch.virginia.edu/fasta_www2/fasta_guide.pdf) 133 1.1 christos ############################################################################### 134 1.1 christos #0 byte 0x3e 135 1.1 christos # q>0 regex =^[>][!-~\t\ ]+$ 136 1.1 christos # Amino Acid codes: [A-IK-Z*-]+ 137 1.1 christos #>>1 regex !=[!-'Jj;:=?@^`|~\\] FASTA 138 1.1 christos # IUPAC codes/gaps: [ACGTURYKMSWBDHVNX-]+ 139 1.1 christos # not in IUPAC codes/gaps: [EFIJLOPQZ] 140 1.1 christos #>>>1 regex !=[EFIJLOPQZefijlopqz] \b, with IUPAC nucleotide codes 141 1.1 christos #>>>1 regex =^[EFIJLOPQZefijlopqz]+$ \b, with Amino Acid codes 142 1.1 christos 143 1.1 christos ############################################################################### 144 1.1 christos # SAM (Sequence Alignment/Map format) 145 1.1 christos # used by SAMtools (http://samtools.sourceforge.net/SAM1.pdf) 146 1.1 christos ############################################################################### 147 1.1 christos # Short-cut version to recognise SAM files with (optional) header at beginning 148 1.1 christos ############################################################################### 149 1.1 christos 0 string @HD\t 150 1.1 christos >4 search VN: Sequence Alignment/Map (SAM), with header 151 1.1 christos >>&0 regex [0-9.]+ \b version %s 152 1.1 christos ############################################################################### 153 1.1 christos # Longer version to recognise SAM alignment lines using (many) regexes 154 1.1 christos ############################################################################### 155 1.1 christos # SAM Alignment QNAME 156 1.1 christos 0 regex =^[!-?A-~]{1,255}(\t[^\t]+){11} 157 1.1 christos # SAM Alignment FLAG 158 1.1 christos >0 regex =^([^\t]+\t){1}[0-9]{1,5}\t 159 1.1 christos # SAM Alignment RNAME 160 1.1 christos >>0 regex =^([^\t]+\t){2}\\*|[^*=]*\t 161 1.1 christos # SAM Alignment POS 162 1.1 christos >>>0 regex =^([^\t]+\t){3}[0-9]{1,9}\t 163 1.1 christos # SAM Alignment MAPQ 164 1.1 christos >>>>0 regex =^([^\t]+\t){4}[0-9]{1,3}\t 165 1.1 christos # SAM Alignment CIGAR 166 1.1 christos >>>>>0 regex =\t(\\*|([0-9]+[MIDNSHPX=])+)\t 167 1.1 christos # SAM Alignment RNEXT 168 1.1 christos >>>>>>0 regex =\t(\\*|=|[!-()+->?-~][!-~]*)\t 169 1.1 christos # SAM Alignment PNEXT 170 1.1 christos >>>>>>>0 regex =^([^\t]+\t){7}[0-9]{1,9}\t 171 1.1 christos # SAM Alignment TLEN 172 1.1 christos >>>>>>>>0 regex =\t[+-]{0,1}[0-9]{1,9}\t.*\t 173 1.1 christos # SAM Alignment SEQ 174 1.1 christos >>>>>>>>>0 regex =^([^\t]+\t){9}(\\*|[A-Za-z=.]+)\t 175 1.1 christos # SAM Alignment QUAL 176 1.1 christos >>>>>>>>>>0 regex =^([^\t]+\t){10}[!-~]+ Sequence Alignment/Map (SAM) 177 1.1 christos >>>>>>>>>>>0 regex =^[@]HD\t.*VN: \b, with header 178 1.1 christos >>>>>>>>>>>>&0 regex =[0-9.]+ \b version %s 179