apraga/org - Change Y4X2CGFKO6ZYMC4MU43CKFQGLHOO45KPCZXXIYD7RL7LYQCVXIQQC

Bisnox : chr21 avec stats presques correctes

Created by Alexis Praga on February 23, 2023

Y4X2CGFKO6ZYMC4MU43CKFQGLHOO45KPCZXXIYD7RL7LYQCVXIQQC

Dependencies

In channels

main

Change contents

Insertion in workout.org at line 5896 [5.1]
[4.50]
```
* <2023-02-24 Fri> Running
25min
```

Replacement in projects/bisonex.org at line 3015 [7.35]

∅:D[2.664] → [6.700:826]

B:BD[8.2254] → [6.700:826]

***** TODO Filtrer variants introniques de référence avec vep
****** TODO variant calling seulf + seulement -f: nombreux FP

[2.664]

[6.826]

******* TODO Filtrer variants introniques de référence avec vep
******** TODO variant calling seulf + seulement -f: nombreux FP

Insertion in projects/bisonex.org at line 3029 [7.35]

[6.2203]

[2.666]


****** TODO Comparer avec stats de NA12878 dans example/happy sur chr21 (exons fournis)
Résultats
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision |
| INDEL | ALL    |        8530 |     8247 |      283 |       14509 |      231 |      5835 |    18 |   141 |      0.966823 |         0.973369 |
| INDEL | PASS   |        8530 |     8247 |      283 |       14483 |      231 |      5809 |    18 |   141 |      0.966823 |         0.973369 |
| SNP   | ALL    |       49913 |    49466 |      447 |       70641 |       69 |     21118 |    40 |     9 |      0.991044 |         0.998607 |
| SNP   | PASS   |       49913 |    49465 |      448 |       70551 |       69 |     21029 |    40 |     9 |      0.991024 |         0.998607 |
Commande
#+begin_src
hap.py pg-hg38.bcf NA12878-GATK3-chr21.vcf.gz -f pg-hg38-conf.bed.gz -r hg38.chr21.fa -o GATK3-xcmp --roc QUAL --roc-filter LowQual
#+end_src

Insertion in projects/bisonex.org at line 3043 [7.35]

[9.114]

[9.377]

En filtrant les exons avec le bed fourni
#+begin_src
 hap.py pg-hg38.bcf NA12878-GATK3-chr21.vcf.gz -f pg-hg38-conf.bed.gz -r hg38.chr21.fa -T exons_hg38.bed.gz -o lol
#+end_src
  Type Filter  TRUTH.TOTAL  TRUTH.TP  TRUTH.FN  QUERY.TOTAL  QUERY.FP  QUERY.UNK  FP.gt  FP.al  METRIC.Recall  METRIC.Precision  METRIC.Frac_NA  METRIC.F1_Score  TRUTH.TOTAL.TiTv_ratio  QUERY.TOTAL.TiTv_ratio  TRUTH.TOTAL.het_hom_ratio  QUERY.TOTAL.het_hom_ratio
INDEL    ALL           97        96         1          138         2         37      0      0       0.989691          0.980198
INDEL   PASS           97        96         1          138         2         37      0      0       0.989691          0.980198
  SNP    ALL          638       636         2          674         0         38      0      0       0.996865          1.000000
  SNP   PASS          638       636         2          674         0         38      0      0       0.996865          1.000000

Replacement in projects/bisonex.org at line 3064 [7.35]
B:BD[10.1476] → [3.77:104]
```
******* DP_over_30 : mieux
```
[10.1476]
[3.104]
```
******* DP
_over_30 : mieux
```
Replacement in projects/bisonex.org at line 3073 [7.35]
B:BD[3.806] → [3.806:810]
```
Ex:
```
[3.806]
[3.810]
```
******** STRT NC_000021.9:45515163 TA -> T : référence non normalisée ??
```

Deletion in projects/bisonex.org at line 3081 [7.35]

∅:D[3.1406] → [10.1476:1626]

B:BD[10.1476] → [10.1476:1626]

******* KILL On réessaye en filtran sur les exons fournis par hap.py
seulement sur le 21...
CLOSED: [2023-02-23 Thu 00:14]
mieux mais beaucoup de FN

Replacement in projects/bisonex.org at line 3082 [7.35]

∅:D[10.1627] → [4.135:927]

B:BD[4.135] → [4.135:927]

***** Comparer avec stats de NA12878 dans example/happy sur chr21
Résultats
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision |
| INDEL | ALL    |        8530 |     8247 |      283 |       14509 |      231 |      5835 |    18 |   141 |      0.966823 |         0.973369 |
| INDEL | PASS   |        8530 |     8247 |      283 |       14483 |      231 |      5809 |    18 |   141 |      0.966823 |         0.973369 |
| SNP   | ALL    |       49913 |    49466 |      447 |       70641 |       69 |     21118 |    40 |     9 |      0.991044 |         0.998607 |
| SNP   | PASS   |       49913 |    49465 |      448 |       70551 |       69 |     21029 |    40 |     9 |      0.991024 |         0.998607 |

[10.1627]

[4.927]

La décomposition de xcmp est
45515163        TAA -> T
45515164        AA -> A
NB: testée avec pre.py
#+begin_src
grep '^#' NA12878_NIST7035_DP_over_30.vcf > NA12878_NIST7035_DP_over_30_chr21.vcf
grep '^NC_000021' NA12878_NIST7035_DP_over_30.vcf >> NA12878_NIST7035_DP_over_30_chr21.vcf

Replacement in projects/bisonex.org at line 3090 [7.35]

B:BD[4.928] → [4.928:937]

Commande

[4.928]

[4.937]

pre.py NA12878_NIST7035_DP_over_30_chr21.vcf out.vcf.gz -r GCA_000001405.15_GRCh38_no_alt_analysis_set.fasta
#+end_src
Selon l'algorithm, on devrait avoir TA -> T et non AA -> A (non left-align)
Si on débug:

Replacement in projects/bisonex.org at line 3095 [7.35]

B:BD[4.949] → [4.949:1081]

hap.py pg-hg38.bcf NA12878-GATK3-chr21.vcf.gz -f pg-hg38-conf.bed.gz -r hg38.chr21.fa -o GATK3-xcmp --roc QUAL --roc-filter LowQual

[4.949]

[4.1081]

grep "^#" NA12878_NIST7035_DP_over_30_chr21.vcf > test_align.vcf
grep 45515163 NA12878_NIST7035_DP_over_30_chr21.vcf  >> test_align.vcf.gz
bgzip test_align.vcf.gz
bcftools index test_align.vcf.gz
multimerge test_align.vcf.gz -o out.vcf.gz -r GCA_000001405.15_GRCh38_no_alt_analysis_set.fasta
#+end_src
Idem... Si on utile un genome de référence plus récent ?
#+begin_src
multimerge test_align.vcf.gz -o out.vcf.gz -r /Work/Groups/bisonex/data/genome/GRCh38.p13/genomeRef.fna --process-full=1

Insertion in projects/bisonex.org at line 3105 [7.35]

[4.1091]

Idem. Et vcfeval ?
#+begin_src
tabix HG001_GRCh38_1_22_v4.2.1_benchmark_chr21.vcf.gz
tabix test_align.vcf.gz
rtg vcfeval -b HG001_GRCh38_1_22_v4.2.1_benchmark_chr21.vcf.gz -c test_align.vcf.gz -t /Work/Groups/bisonex/data/genome/GRCh38.p13/genomeRef.sdf -o vcfeval-test
#+end_src

Replacement in projects/bisonex.org at line 3112 [7.35]

B:BD[4.1092] → [4.1092:1133]

En filtrant les exons avec le bed fourni

[4.1092]

[4.1133]

Selected score threshold using: maximized F-measure
Threshold  True-pos-baseline  True-pos-call  False-pos  False-neg  Precision  Sensitivity  F-measure
----------------------------------------------------------------------------------------------------
   99.000                  0              0          1      54828     0.0000       0.0000     0.0000
     None                  0              0          1      54828     0.0000       0.0000     0.0000
     On essaie les différentes étapes

Replacement in projects/bisonex.org at line 3120 [7.35]

B:BD[4.1145] → [4.1145:1260]

 hap.py pg-hg38.bcf NA12878-GATK3-chr21.vcf.gz -f pg-hg38-conf.bed.gz -r hg38.chr21.fa -T exons_hg38.bed.gz -o lol

[4.1145]

[4.1260]

multimerge test_align.vcf.gz -o out.vcf.gz -r /Work/Groups/bisonex/data/genome/GRCh38.p13/genomeRef.fna --homref-split 1 --homref-vcf-out 1 --trimalleles 1 --splitalleles 1

Insertion in projects/bisonex.org at line 3122 [7.35]
[4.1270]
[4.1270]
```
1er changement
TAA T
TAA TA
```

Replacement in projects/bisonex.org at line 3126 [7.35]

B:BD[4.1271] → [4.1271:2042]

  Type Filter  TRUTH.TOTAL  TRUTH.TP  TRUTH.FN  QUERY.TOTAL  QUERY.FP  QUERY.UNK  FP.gt  FP.al  METRIC.Recall  METRIC.Precision  METRIC.Frac_NA  METRIC.F1_Score  TRUTH.TOTAL.TiTv_ratio  QUERY.TOTAL.TiTv_ratio  TRUTH.TOTAL.het_hom_ratio  QUERY.TOTAL.het_hom_ratio
INDEL    ALL           97        96         1          138         2         37      0      0       0.989691          0.980198
INDEL   PASS           97        96         1          138         2         37      0      0       0.989691          0.980198
  SNP    ALL          638       636         2          674         0         38      0      0       0.996865          1.000000
  SNP   PASS          638       636         2          674         0         38      0      0       0.996865          1.000000

[4.1271]

[4.2042]

Après réflexion, c'est la référence qui n'est pas normalisée ! (left-trimmed)
******** NC_000021.9     14108836  T -> C
Dans le bam mais filtré par DP
******* variant calling seul : meilleur score pour l'instant
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision
| INDEL | ALL    |          76 |       43 |       33 |          82 |       18 |        20 |     3 |     5 |      0.565789 |         0.709677
| INDEL | PASS   |          76 |       43 |       33 |          82 |       18 |        20 |     3 |     5 |      0.565789 |         0.709677
| SNP   | ALL    |         582 |      448 |      134 |         530 |       25 |        57 |     6 |     1 |      0.769759 |         0.947146
| SNP   | PASS   |         582 |      448 |      134 |         530 |       25 |        57 |     6 |     1 |      0.769759 |         0.947146

Insertion in projects/bisonex.org at line 3136 [7.35]

[4.2043]

[11.818]

 | METRIC.Frac_NA | METRIC.F1_Score | TRUTH.TOTAL.TiTv_ratio | QUERY.TOTAL.TiTv_ratio | TRUTH.TOTAL.het_hom_ratio | QUERY.TOTAL.het_hom_ratio |
 |       0.243902 |        0.629617 |                        |                        |        1.1818181818181819 |        1.6129032258064515 |
 |       0.243902 |        0.629617 |                        |                        |        1.1818181818181819 |        1.6129032258064515 |
 |       0.107547 |        0.849289 |     3.0985915492957745 |      2.925925925925926 |        1.5304347826086957 |         1.774869109947644 |
 |       0.107547 |        0.849289 |     3.0985915492957745 |      2.925925925925926 |        1.5304347826086957 |         1.774869109947644 |

Insertion in projects/bisonex.org at line 3143 [7.35]
[11.890]
[12.96992]