apraga/org - Change PDH2BEBXR6WCCO2GRS3L6HMLQNCC2JU5BOKZLV4DLFPQD2UZFJKQC

Generating all notes with hakyll (dumping org-roam)

Created by Alexis Praga on November 25, 2023

PDH2BEBXR6WCCO2GRS3L6HMLQNCC2JU5BOKZLV4DLFPQD2UZFJKQC

Dependencies

In channels

main

Change contents

File deletion: site.tar.gz
BF:BFD[5.2] → [6.127970:128012]
BF:BFD[6.128012] → [6.389:389]
B:BD[6.389] → [6.390:127969]

Insertion in src/Main.hs at line 10 [8.9907]

[7.1238]

notes = fromList $ ["notes/index.org" , "notes/japonais.org"]

Insertion in src/Main.hs at line 33 [8.9907]

[9.220]

            >>= relativizeUrls
    match notes $ do
        route $ setExtension "html"
        compile $ pandocCompiler
            >>= relativizeUrls
    match "notes/medecine/*.org" $ do
        route $ setExtension "html"
        compile $ pandocCompiler

Insertion in src/Main.hs at line 45 [8.9907]
[9.252]
[10.789]
Replacement in projects.org at line 77 [12.123895]
B:BD[11.324] → [4.15:43]
```
SCHEDULED: <2023-11-24 Fri>
```
[11.324]
[11.352]
```
SCHEDULED: <2023-11-28 Tue>
```
Replacement in projects.org at line 82 [12.123895]
B:BD[13.73] → [13.73:132]
```
CLOSED: [2023-11-22 Wed 20:41] SCHEDULED: <2023-11-25 Sat>
```
[13.73]
[14.106]
```
SCHEDULED: <2023-11-23 Thu>
```

Insertion in projects.org at line 88 [12.123895]

[13.253]

[15.614]

** DONE Demande remboursement essence formation IH
CLOSED: [2023-11-25 Sat 11:26] SCHEDULED: <2023-11-24 Fri>
/Entered on/ [2023-11-23 Thu 22:56]
- EFS: non
- CHU : renvoie vers trevenans
** Hémato
:PROPERTIES:
:CATEGORY: hemato
:END:
*** TODO Cours UNESS
SCHEDULED: <2023-11-25 Sat 19:00>

Replacement in projects.org at line 104 [12.123895]
B:BD[15.705] → [4.44:77]
```
SCHEDULED: <2023-11-24 Fri .+1d>
```
[15.705]
[15.738]
```
SCHEDULED: <2023-11-25 Sat .+1d>
```

Insertion in projects.org at line 109 [12.123895]

[15.811]

[4.116]

- State "DONE"       from "TODO"       [2023-11-24 Fri 22:13]

Replacement in projects.org at line 398 [12.123895]

B:BD[15.1056] → [4.891:976]

**** TODO Correction réponse reviewer +/ article (gain)
SCHEDULED: <2023-11-23 Thu>

[15.1056]

[4.976]

**** DONE Correction réponse reviewer +/ article (gain)
CLOSED: [2023-11-23 Thu 22:57] SCHEDULED: <2023-11-23 Thu>

Replacement in projects.org at line 422 [12.123895]

B:BD[11.745] → [11.745:800]

SCHEDULED: <2023-11-24 Fri> DEADLINE: <2023-12-04 Mon>

[11.745]

[11.800]

SCHEDULED: <2023-11-25 Sat 15:00> DEADLINE: <2023-12-04 Mon>

Insertion in projects.org at line 700 [12.123895]

[4.1146]

[16.1513]

** TODO Command minoxidil
SCHEDULED: <2023-11-25 Sat 17:00>
/Entered on/ [2023-11-25 Sat 11:30]

Replacement in projects/bisonex.org at line 30 [17.35]

B:BD[11.17629] → [11.17629:25760]

∅:D[11.25760] → [15.18505:18566]

B:BD[15.18505] → [15.18505:18566]


*** DONE Biblio performance aligneur <(biblio aligneur)> <(aligneur)>
CLOSED: [2023-10-13 Fri 17:40] SCHEDULED: <2023-10-01 Sun>
*** DONE Figure: nombre d'articles citant les principaux aligneur par année
CLOSED: [2023-10-11 Wed 23:54] SCHEDULED: <2023-10-03 Tue>
Il faudrait utiliser pubmed en local, sinon c'est 10 000 requete par aligner !
*** DONE Figure: nombre d'articles citant les principaux aligneur
CLOSED: [2023-10-12 Thu 23:58] SCHEDULED: <2023-10-12 Thu>
Il faudrait utiliser pubmed en local, sinon c'est 10 000 requete par aligner !
On se base sur
** Appel de variant
*** TODO Biblio <(biblio appel variant)> <(appel variant)>
SCHEDULED: <2023-11-22 Wed>
*** TODO Figure: nombre de publication par appel de variant
SCHEDULED: <2023-11-07 Tue>
/Entered on/ [2023-09-19 Tue 08:43]
** TODO Figure: nombre d'exomes par années
SCHEDULED: <2023-11-26 Sun>
/Entered on/ [2023-09-19 Tue 08:43]
* Tests :tests:
** KILL Non régression : version prod
CLOSED: [2023-05-23 Tue 08:46]
*** DONE ID common snp
CLOSED: [2022-11-19 Sat 21:36]
#+begin_src
$ wc -l ID_of_common_snp.txt
23194290 ID_of_common_snp.txt
$ wc -l /Work/Users/apraga/bisonex/database/dbSNP/ID_of_common_snp.txt
23194290 /Work/Users/apraga/bisonex/database/dbSNP/ID_of_common_snp.txt
#+end_src
*** DONE ID common snp not clinvar patho
CLOSED: [2022-12-11 Sun 20:11]
**** DONE Vérification du problème
CLOSED: [2022-12-11 Sun 16:30]
Sur le J:
21155134 /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt.ref
Version de "non-régression"
21155076 database/dbSNP/ID_of_common_snp_not_clinvar_patho.txt
Nouvelle version
23193391 /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt
Si on enlève les doublons
$ sort database/dbSNP/ID_of_common_snp_not_clinvar_patho.txt | uniq > old.txt
$ wc -l old.txt
21107097 old.txt
$ sort /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt | uniq > new.txt
$ wc -l new.txt
21174578 new.txt
$ sort /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt.ref | uniq > ref.txt
$ wc -l ref.txt
21107155 ref.txt
Si on regarde la différence
 comm -23 ref.txt old.txt
rs1052692
rs1057518973
rs1057518973
rs11074121
rs112848754
rs12573787
rs145033890
rs147889095
rs1553904159
rs1560294695
rs1560296615
rs1560310926
rs1560325547
rs1560342418
rs1560356225
rs1578287542
...
On cherche le premier
bcftools query -i 'ID="rs1052692"' database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 1619351 C A,T
Il est bien patho...
$ bcftools query -i 'POS=1619351' database/clinvar/clinvar.vcf.gz -f '%CHROM %POS %REF %ALT %INFO/CLNSIG\n'
19 1619351 C T Conflicting_interpretations_of_pathogenicity
On vérifie pour tous les autres
$ comm -23 ref.txt old.txt > tocheck.txt
On génère les régions à vérifier (chromosome number:position)
$ bcftools query -i 'ID=@tocheck.txt' database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM\t%POS\n' > tocheck.pos
On génère le mapping inverse (chromosome number -> NC)
$ awk ' { t = $1; $1 = $2; $2 = t; print; } ' database/RefSeq/refseq_to_number_only_consensual.txt  > mapping.txt
On remap clinvar
$ bcftools annotate --rename-chrs mapping.txt database/clinvar/clinvar.vcf.gz -o clinvar_remapped.vcf.gz
$ tabix clinvar_remapped.vcf.gz
Enfin, on cherche dans clinvar la classification
$ bcftools query -R tocheck.pos clinvar_remapped.vcf.gz -f '%CHROM %POS %INFO/CLNSIG\n'
$ bcftools query -R tocheck.pos database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM %POS %ID \n' | grep '^NC'
#+RESULTS:
**** DONE Comprendre pourquoi la nouvelle version donne un résultat différent
CLOSED: [2022-12-11 Sun 20:11]
***** DONE Même version dbsnp et clinvar ?
CLOSED: [2022-12-10 Sat 23:02]
Clinvar différent !
  $ bcftools stats clinvar.gz
  clinvar (Alexis)
SN	0	number of samples:	0
SN	0	number of records:	1492828
SN	0	number of no-ALTs:	965
SN	0	number of SNPs:	1338007
SN	0	number of MNPs:	5562
SN	0	number of indels:	144580
SN	0	number of others:	3714
SN	0	number of multiallelic sites:	0
SN	0	number of multiallelic SNP sites:	0
clinvar (new)
SN	0	number of samples:	0
SN	0	number of records:	1493470
SN	0	number of no-ALTs:	965
SN	0	number of SNPs:	1338561
SN	0	number of MNPs:	5565
SN	0	number of indels:	144663
SN	0	number of others:	3716
SN	0	number of multiallelic sites:	0
SN	0	number of multiallelic SNP sites:	0
***** DONE Mettre à jour clinvar et dbnSNP pour travailler sur les mêm bases
CLOSED: [2022-12-11 Sun 12:10]
Problème persiste
***** DONE Supprimer la conversion en int du chromosome
CLOSED: [2022-12-10 Sat 19:29]
***** KILL Même NC ?
CLOSED: [2022-12-10 Sat 19:29]
$  zgrep "contig=<ID=NC_\(.*\)" clinvar/GRCh38/clinvar.vcf.gz > contig.clinvar
$ diff contig.txt contig.clinvar
< ##contig=<ID=NC_012920.1>
***** DONE Tester sur chromosome 19: ok
CLOSED: [2022-12-11 Sun 13:53]
On prépare les données
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/dbSNP_common.vcf.gz -o dbSNP_common_19.vcf.gz
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data/clinvar/GRCh38/clinvar.vcf.gz -o clinvar_19.vcf.gz
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data-alexis/dbSNP/dbSNP_common.vcf.gz -o dbSNP_common_19_old.vcf.gz
 bcftools filter -i 'CHROM="19"' /Work/Groups/bisonex/data-alexis/clinvar/clinvar.vcf.gz -o clinvar_19_old.vcf.gz
#+end_src
On récupère les 2 versions du script
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
git checkout regression ../../script/pythonScript/clinvar_sbSNP.py
cp ../../script/pythonScript/clinvar_sbSNP.py clinvar_sbSNP_old.py
git checkout HEAD ../../script/pythonScript/clinvar_sbSNP.py
#+end_src
#+RESULTS:
On compare
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
python ../../script/pythonScript/clinvar_sbSNP.py clinvar_sbSNP.py --clinvar clinvar_19.vcf.gz --dbSNP dbSNP_common_19.vcf.gz --output tmp.txt
sort tmp.txt | uniq > new.txt
table=/Work/Groups/bisonex/data-alexis/RefSeq/refseq_to_number_only_consensual.txt
python clinvar_sbSNP_old.py --clinvar clinvar_19_old.vcf.gz --dbSNP dbSNP_common_19_old.vcf.gz --output tmp_old.txt --chrm_name_table $table
sort tmp_old.txt | uniq > old.txt
wc -l old.txt new.txt
#+end_src
#+RESULTS:
|  535155 | old.txt |
|  535194 | new.txt |
| 1070349 | total   |
Si on prend le premier manquant dans new, il est conflicting patho donc il ne devrait pas y être...
$ bcftools query -i 'ID="rs10418277"' dbSNP
_common_19.vcf.gz  -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 54939682 C G,T
$ bcftools query -i 'ID="rs10418277"' dbSNP_common_19_old.vcf.gz  -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 54939682 C G,T
$ bcftools query -i 'POS=54939682' clinvar_19.vcf.gz  -f '%POS %REF %ALT %INFO/CLNSIG\n'
54939682 C G Conflicting_interpretations_of_pathogenicity
54939682 C T Benign
$ bcftools query -i 'POS=54939682' clinvar_19_old.vcf.gz  -f '%POS %REF %ALT %INFO/CLNSIG\n'
54939682 C G Conflicting_interpretations_of_pathogenicity
54939682 C T Benign
$ grep rs10418277 *.txt
new.txt:rs10418277
tmp.txt:rs10418277
Le problème venait de la POS qui n'était plus convertie en int (suppression de la ligne par erreur ??)
On vérifie
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
python ../../script/pythonScript/clinvar_sbSNP.py --clinvar clinvar_19.vcf.gz --dbSNP dbSNP_common_19.vcf.gz --output tmp.txt
sort tmp.txt | uniq > new.txt
table=/Work/Groups/bisonex/data-alexis/RefSeq/refseq_to_number_only_consensual.txt
python clinvar_sbSNP_old.py --clinvar clinvar_19_old.vcf.gz --dbSNP dbSNP_common_19_old.vcf.gz --output tmp_old.txt --chrm_name_table $table
sort tmp_old.txt | uniq 
> old.txt
wc -l old.txt new.txt
diff old.txt new.txt
#+end

[11.17629]

[15.18566]


*** DONE Biblio performance aligneur <(biblio aligneur)> <(aligneur)>
CLOSED: [2023-10-13 Fri 17:40] SCHEDULED: <2023-10-01 Sun>
*** DONE Figure: nombre d'articles citant les principaux aligneur par année
CLOSED: [2023-10-11 Wed 23:54] SCHEDULED: <2023-10-03 Tue>
Il faudrait utiliser pubmed en local, sinon c'est 10 000 requete par aligner !
*** DONE Figure: nombre d'articles citant les principaux aligneur
CLOSED: [2023-10-12 Thu 23:58] SCHEDULED: <2023-10-12 Thu>
Il faudrait utiliser pubmed en local, sinon c'est 10 000 requete par aligner !
On se base sur
** Appel de variant
*** TODO Biblio <(biblio appel variant)> <(appel variant)>
SCHEDULED: <2023-11-25 Sat 11:00>
*** TODO Figure: nombre de publication par appel de variant
SCHEDULED: <2023-11-07 Tue>
/Entered on/ [2023-09-19 Tue 08:43]
** TODO Figure: nombre d'exomes par années
SCHEDULED: <2023-11-26 Sun>
/Entered on/ [2023-09-19 Tue 08:43]
* Tests :tests:
** KILL Non régression : version prod
CLOSED: [2023-05-23 Tue 08:46]
*** DONE ID common snp
CLOSED: [2022-11-19 Sat 21:36]
#+begin_src
$ wc -l ID_of_common_snp.txt
23194290 ID_of_common_snp.txt
$ wc -l /Work/Users/apraga/bisonex/database/dbSNP/ID_of_common_snp.txt
23194290 /Work/Users/apraga/bisonex/database/dbSNP/ID_of_common_snp.txt
#+end_src
*** DONE ID common snp not clinvar patho
CLOSED: [2022-12-11 Sun 20:11]
**** DONE Vérification du problème
CLOSED: [2022-12-11 Sun 16:30]
Sur le J:
21155134 /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt.ref
Version de "non-régression"
21155076 database/dbSNP/ID_of_common_snp_not_clinvar_patho.txt
Nouvelle version
23193391 /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt
Si on enlève les doublons
$ sort database/dbSNP/ID_of_common_snp_not_clinvar_patho.txt | uniq > old.txt
$ wc -l old.txt
21107097 old.txt
$ sort /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt | uniq > new.txt
$ wc -l new.txt
21174578 new.txt
$ sort /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/ID_of_common_snp_not_clinvar_patho.txt.ref | uniq > ref.txt
$ wc -l ref.txt
21107155 ref.txt
Si on regarde la différence
 comm -23 ref.txt old.txt
rs1052692
rs1057518973
rs1057518973
rs11074121
rs112848754
rs12573787
rs145033890
rs147889095
rs1553904159
rs1560294695
rs1560296615
rs1560310926
rs1560325547
rs1560342418
rs1560356225
rs1578287542
...
On cherche le premier
bcftools query -i 'ID="rs1052692"' database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 1619351 C A,T
Il est bien patho...
$ bcftools query -i 'POS=1619351' database/clinvar/clinvar.vcf.gz -f '%CHROM %POS %REF %ALT %INFO/CLNSIG\n'
19 1619351 C T Conflicting_interpretations_of_pathogenicity
On vérifie pour tous les autres
$ comm -23 ref.txt old.txt > tocheck.txt
On génère les régions à vérifier (chromosome number:position)
$ bcftools query -i 'ID=@tocheck.txt' database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM\t%POS\n' > tocheck.pos
On génère le mapping inverse (chromosome number -> NC)
$ awk ' { t = $1; $1 = $2; $2 = t; print; } ' database/RefSeq/refseq_to_number_only_consensual.txt  > mapping.txt
On remap clinvar
$ bcftools annotate --rename-chrs mapping.txt database/clinvar/clinvar.vcf.gz -o clinvar_remapped.vcf.gz
$ tabix clinvar_remapped.vcf.gz
Enfin, on cherche dans clinvar la classification
$ bcftools query -R tocheck.pos clinvar_remapped.vcf.gz -f '%CHROM %POS %INFO/CLNSIG\n'
$ bcftools query -R tocheck.pos database/dbSNP/dbSNP_common.vcf.gz -f '%CHROM %POS %ID \n' | grep '^NC'
#+RESULTS:
**** DONE Comprendre pourquoi la nouvelle version donne un résultat différent
CLOSED: [2022-12-11 Sun 20:11]
***** DONE Même version dbsnp et clinvar ?
CLOSED: [2022-12-10 Sat 23:02]
Clinvar différent !
  $ bcftools stats clinvar.gz
  clinvar (Alexis)
SN	0	number of samples:	0
SN	0	number of records:	1492828
SN	0	number of no-ALTs:	965
SN	0	number of SNPs:	1338007
SN	0	number of MNPs:	5562
SN	0	number of indels:	144580
SN	0	number of others:	3714
SN	0	number of multiallelic sites:	0
SN	0	number of multiallelic SNP sites:	0
clinvar (new)
SN	0	number of samples:	0
SN	0	number of records:	1493470
SN	0	number of no-ALTs:	965
SN	0	number of SNPs:	1338561
SN	0	number of MNPs:	5565
SN	0	number of indels:	144663
SN	0	number of others:	3716
SN	0	number of multiallelic sites:	0
SN	0	number of multiallelic SNP sites:	0
***** DONE Mettre à jour clinvar et dbnSNP pour travailler sur les mêm bases
CLOSED: [2022-12-11 Sun 12:10]
Problème persiste
***** DONE Supprimer la conversion en int du chromosome
CLOSED: [2022-12-10 Sat 19:29]
***** KILL Même NC ?
CLOSED: [2022-12-10 Sat 19:29]
$  zgrep "contig=<ID=NC_\(.*\)" clinvar/GRCh38/clinvar.vcf.gz > contig.clinvar
$ diff contig.txt contig.clinvar
< ##contig=<ID=NC_012920.1>
***** DONE Tester sur chromosome 19: ok
CLOSED: [2022-12-11 Sun 13:53]
On prépare les données
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data/dbSNP/GRCh38.p13/dbSNP_common.vcf.gz -o dbSNP_common_19.vcf.gz
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data/clinvar/GRCh38/clinvar.vcf.gz -o clinvar_19.vcf.gz
bcftools filter -i 'CHROM="NC_000019.10"' /Work/Groups/bisonex/data-alexis/dbSNP/dbSNP_common.vcf.gz -o dbSNP_common_19_old.vcf.gz
 bcftools filter -i 'CHROM="19"' /Work/Groups/bisonex/data-alexis/clinvar/clinvar.vcf.gz -o clinvar_19_old.vcf.gz
#+end_src
On récupère les 2 versions du script
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
git checkout regression ../../script/pythonScript/clinvar_sbSNP.py
cp ../../script/pythonScript/clinvar_sbSNP.py clinvar_sbSNP_old.py
git checkout HEAD ../../script/pythonScript/clinvar_sbSNP.py
#+end_src
#+RESULTS:
On compare
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
python ../../script/pythonScript/clinvar_sbSNP.py clinvar_sbSNP.py --clinvar clinvar_19.vcf.gz --dbSNP dbSNP_common_19.vcf.gz --output tmp.txt
sort tmp.txt | uniq > new.txt
table=/Work/Groups/bisonex/data-alexis/RefSeq/refseq_to_number_only_consensual.txt
python clinvar_sbSNP_old.py --clinvar clinvar_19_old.vcf.gz --dbSNP dbSNP_common_19_old.vcf.gz --output tmp_old.txt --chrm_name_table $table
sort tmp_old.txt | uniq > old.txt
wc -l old.txt new.txt
#+end_src
#+RESULTS:
|  535155 | old.txt |
|  535194 | new.txt |
| 1070349 | total   |
Si on prend le premier manquant dans new, il est conflicting patho donc il ne devrait pas y être...
$ bcftools query -i 'ID="rs10418277"' dbSNP
_common_19.vcf.gz  -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 54939682 C G,T
$ bcftools query -i 'ID="rs10418277"' dbSNP_common_19_old.vcf.gz  -f '%CHROM %POS %REF %ALT\n'
NC_000019.10 54939682 C G,T
$ bcftools query -i 'POS=54939682' clinvar_19.vcf.gz  -f '%POS %REF %ALT %INFO/CLNSIG\n'
54939682 C G Conflicting_interpretations_of_pathogenicity
54939682 C T Benign
$ bcftools query -i 'POS=54939682' clinvar_19_old.vcf.gz  -f '%POS %REF %ALT %INFO/CLNSIG\n'
54939682 C G Conflicting_interpretations_of_pathogenicity
54939682 C T Benign
$ grep rs10418277 *.txt
new.txt:rs10418277
tmp.txt:rs10418277
Le problème venait de la POS qui n'était plus convertie en int (suppression de la ligne par erreur ??)
On vérifie
#+begin_src sh :dir /ssh:meso:/Work/Users/apraga/bisonex/tests/debug-commonsnp
PATH=$PATH:$HOME/.nix-profile/bin
python ../../script/pythonScript/clinvar_sbSNP.py --clinvar clinvar_19.vcf.gz --dbSNP dbSNP_common_19.vcf.gz --output tmp.txt
sort tmp.txt | uniq > new.txt
table=/Work/Groups/bisonex/data-alexis/RefSeq/refseq_to_number_only_consensual.txt
python clinvar_sbSNP_old.py --clinvar clinvar_19_old.vcf.gz --dbSNP dbSNP_common_19_old.vcf.gz --output tmp_old.txt --chrm_name_table $table
sort tmp_old.txt | uniq > old.txt
wc -l old.txt new.txt
diff old.txt new.txt
#+end

Replacement in projects/bisonex.org at line 63 [17.35]

B:BD[18.49719] → [13.420:8612]

∅:D[13.8612] → [19.8851:8852]

∅:D[20.25350] → [19.8851:8852]

B:BD[14.8506] → [19.8851:8852]

B:BD[19.8852] → [15.18568:26698]

B:BD[15.26698] → [11.25761:25822]

B:BD[11.25822] → [4.9355:17547]

    0.529245 |
Hg38
| Type  | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision |
| INDEL |         549 |      489 |       60 |         899 |       64 |       340 |     8 |    17 |      0.890710 |         0.885510 |
| SNP   |       21973 |    21462 |      511 |       26285 |      563 |      4263 |    68 |    16 |      0.976744 |         0.974435 |
****** DONE Interesection des bed: similaire
CLOSED: [2023-07-04 Tue 23:11]
HG38
 #+begin_src sh
 bedtools intersect -a capture/Agilent_SureSelect_All_Exons_v7_hg38_Regions.bed -b /Work/Groups/bisonex/data/giab/GRCh38/HG001_GRCh38_1_22_v4.2.1_benchmark.bed  | wc -l
 #+end_src
 204280
 T2T
 #+begin_src sh
 bedtools intersect -a /Work/Groups/bisonex/data/giab/T2T/Agilent_SureSelect_All_Exons_v7_hg38_Regions_hg38_T2T.bed -b /Work/Groups/bisonex/data/giab/T2T/HG001_GRCh38_1_22_v4.2.1_benchmark_hg38_T2T.bed  | wc -l
 #+end_src
 204021
****** DONE Vérifier la ligne de commande
CLOSED: [2023-07-04 Tue 23:38]
#+begin_src sh
hap.py \
    HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz \
    HG001-SRX11061486_SRR14724513-T2T.vcf.gz \
     \
    --reference chm13v2.0.fa \
    --threads 6 \
     \
    -T Agilent_SureSelect_All_Exons_v7_hg38_Regions_hg38_T2T.bed \
    --false-positives HG001_GRCh38_1_22_v4.2.1_benchmark_hg38_T2T.bed \
     \
    -o HG001
#+end_src
****** DONE Corriger FILTER : mieux mais toujours trop de négatifs. 3/4 SNP retrouvés
CLOSED: [2023-07-08 Sat 15:19] SCHEDULED: <2023-07-08 Sat>
 Type Filter  TRUTH.TOTAL  TRUTH.TP  TRUTH.FN  QUERY.TOTAL  QUERY.FP  QUERY.UNK  FP.gt  FP.al  METRIC.Recall  METRIC.Precision  METRIC.Frac_NA  METRIC.F1_Score  TRUTH.TOTAL.TiTv_ratio  QUERY.TOTAL.TiTv_ratio  TRUTH.TOTAL.het_hom_ratio  QUERY.TOTAL.het_hom_ratio
INDEL    ALL          413       246       167          751       289        215      2     98       0.595642          0.460821        0.286285         0.519629                     NaN                     NaN                   2.428571                   2.465116
INDEL   PASS          413       246       167          751       289        215      2     98       0.595642          0.460821        0.286285         0.519629                     NaN                     NaN                   2.428571                   2.465116
  SNP    ALL        15883     15479       404        23597      5277       2841     46     44       0.974564          0.745760        0.120397         0.844947                3.017198                 2.85705                   5.560099                   2.114633
  SNP   PASS        15883     15479       404        23597      5277       2841     46     44       0.974564          0.745760        0.120397         0.844947                3.017198                 2.85705                   5.560099                   2.114633
******* DONE Vérifier qu'il ne reste plus de filtre autre que PASS
CLOSED: [2023-07-08 Sat 15:19]
#+begin_src
$ zgrep -c 'PASS' HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz
3730505
$ zgrep -c '^chr' HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz
3730506
#+end_src
****** TODO 1/4 SNP manquant ?
******* DONE Regarder avec Julia si ce sont vraiment des FP: 61/5277 qui ne le sont pas
CLOSED: [2023-07-09 Sun 12:09]
******* DONE Examiner les FP
CLOSED: [2023-07-30 Sun 22:05]
******* DONE Tester un FP
CLOSED: [2023-07-30 Sun 22:05]
  2 │ chr1        608765  A           G           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:ti:SNP:homalt:188
  liftDown UCSC: rien en GIAB : vrai FP
 3 │ chr1        762943  A           G           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:ti:SNP:homalt:287
 4 │ chr1        762945  A           T           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:tv:SNP:homalt:287
 Remaniements complexes ? Pas dans le gène en HG38
******* DONE La plupart des FP (4705/5566) sont homozygotes: erreur de référence ?
CLOSED: [2023-07-12 Wed 21:10] SCHEDULED: <2023-07-09 Sun>
Sur les 2 premiers variants, ils montrent en fait la différence entre T2T et GRCh38
Erreur à l'alignement ?
******** KILL relancer l'alignement
CLOSED: [2023-07-09 Sun 17:36]
******** DONE vérifier reads identiques hg38 et T2T: oui
CLOSED: [2023-07-09 Sun 16:36]
T2T CHR1608765
38   	chr1:1180168-1180168 (
SRR14724513.24448214
SRR14724513.24448214
******* DONE Vérifier quelques variants sur IGV
CLOSED: [2023-07-09 Sun 17:36]
******* KILL Répartition des FP : cluster ?
CLOSED: [2023-07-09 Sun 17:36]
****** DONE Examiner les FP restant après correction selon séquence de référence
CLOSED: [2023-08-12 Sat 15:57]
****** HOLD Examiner les variants supprimé
****** TODO Enlever les FP qui correspondent à un changement dans le génome
******* Condition:
- pas de variation à la position en GRCh38
- variantion homozygote
- la varation en T2T correspond au changement de pair de base GRC38 -> T2T
  pour les SNP:
  alt_T2T[i] = DNA_GRC38[j]
  avec i la position en T2T et j la position en GRCh38
  Note: définir un ID n'est pas correct car les variants peuvent être modifié par happy !
******* Idée
 - Pour chaque FP, c'est un "faux" FP si
     - REF en hg38 == ALT en T2T
     - et REF en hg38 != REF en T2T
     - et variant homozygote
Comment obtenir les séquences de réferences ?
1. liftover
2. blat sur la séquence autour du variant
3. identifier quelques reads contenant le variant et regarder leur aligneement en hg38
Après discussion avec Alexis: solution 3
******* Algorithme
1. Extraire les coordonnées en T2T des faux positifs *homozygote*
2. Pour chaque faux positif
   1. lister 10 reads contenant le variant
   2. pour chacun de ces reads, récupérer la séquence en T2T et GRCh38 via le nom du read dans le bam
   3. si la séquence en T2T modifiée par le variant est "identique" à celle en GRCh38, alors on ignore ce faux positif
Note: on ignore les reads qui ont changé de chromosome entre les version
******* DONE Résultat préliminaire
CLOSED: [2023-07-23 Sun 14:30]
cf [[file:~/roam/research/bisonex/code/giab/giab-corrected.csv][script julia]]
3498 faux positifs en moins, soit 0.89 sensibilité
julia> tp=15479
julia> fp=5277
julia> tp/(tp+fp)
0.7457602620928888
julia> tp/(tp+(fp-3498))
0.8969173716537258
On est toujours en dessous des 97%
******* HOLD Corriger proprement VCF ou résultats Happy
******* TODO Adapter pour gérer plusieurs variants par read
****** DONE Méthodologie du pangenome
CLOSED: [2023-10-03 Tue 21:28]
Voir biblio[cite:@liao2023]  mais ont aligné sur GRCH38
******* DONE Mail alexis
CLOSED: [2023-10-03 Tue 21:28]
****** DONE Méthodologie T2T
CLOSED: [2023-10-16 Mon 19:42]
Mail alexis
SCHEDULED: <2023-10-04 Wed>
***** TODO Rendre simplement le nombre de vrais positifs
SCHEDULED: <2023-11-25 Sat>
***** KILL Mail Yannis
CLOSED: [2023-07-08 Sat 10:44]
***** DONE Mail GIAB pour version T2T
CLOSED: [2023-07-07 Fri 18:37]
**** TODO HG002 :hg002:T2T:
**** TODO HG003 :hg003:T2T:
**** TODO HG004 :hg004:T2T:
**** DONE Plot : ashkenazim trio :hg38:
CLOSED: [2023-07-30 Sun 16:49] SCHEDULED: <2023-07-30 Sun 15:00>
:LOGBOOK:
CLOCK: [2023-07-30 Sun 16:06]--[2023-07-30 Sun 16:35] =>  0:29
CLOCK: [2023-07-30 Sun 15:39]--[2023-07-30 Sun 15:40] =>  0:01
:END:
/Entered on/ [2023-04-16 Sun 17:29]
Refaire résultats
**** DONE Mail Paul sur les résultat ashkenazim +/- centogene
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** DONE Relancer comparaison GIAB avec GATK 4.4.0
CLOSED: [2023-08-12 Sat 15:55]
/Entered on/ [2023-08-03 Thu 12:42]
*** TODO Platinum genome :platinum:
https://emea.illumina.com/platinumgenomes.html
**** TODO Tester sur la zone couverte par l'exome centogène
SCHEDULED: <2023-11-25 Sat>
*** DONE Séquencer NA12878 :cento:hg001:
CLOSED: [2023-10-07 Sat 17:59]
Discussion avec Paul : sous-traitant ne nous donnera pas les données, il faut commander l'ADN
**** DONE ADN commandé
CLOSED: [2023-06-30 Fri 22:29]
**** DONE Sauvegarder les données brutes
CLOSED: [2023-07-30 Sun 14:22] SCHEDULED: <2023-07-19 Wed>
K, scality, S
**** KILL Récupérer le fichier de capture
CLOSED: [20
2
3-07-30 Sun 14:25] SCHEDULED: <2023-07-23 Sun>
Candidats donnés dans publication https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8354858/
#+begin_quote
In short, the Nextera Rapid Capture Exome Kit (Illumina, San Diego, CA), the SureSelect Human All Exon kit (Agilent, Santa Clara, CA) or the Twist Human Core Exome was used for enrichment, and a Nextseq500, HiSeq4000, or Novoseq 6000 (Illumina) instrument was used for the actual sequencing, with the average coverage targeted to at least 100× or at least 98% of the target DNA covered 20×.
#+end_quote
Par défaut, on utilisera https://www.twistbioscience.com/products/ngs/alliance-panels#tab-3
ANnonce récente pour nouveau panel Twist : https://www.centogene.com/news-events/news/newsdetails/twist-bioscience-and-centogene-launch-three-panels-to-advance-rare-disease-and-hereditary-cancer-research-and-support-diagnostics
Masi pas de fichier BED
***** DONE Mail centogène
CLOSED: [2023-07-30 Sun 14:22] DEADLINE: <2023-07-23 Sun>
**** DONE Tester Nextera Rapid Capture Exome v1.2 (hg19) :giab:
CLOSED: [2023-08-06 Sun 19:05] SCHEDULED: <2023-08-03 Thu 19:00>
https://support.illumina.com/downloads/nextera-rapid-capture-exome-v1-2-product-files.html
***** DONE Liftover capture
CLOSED: [2023-08-06 Sun 18:30] SCHEDULED: <2023-08-06 Sun>
#+begin_src sh
 nextflow run -profile standard,helios workflows/lift-nextera-capture.nf  -lib lib
#+end_src
Vérification rapide : ok
***** DONE Run
CLOSED: [2023-08-06 Sun 19:05] SCHEDULED: <2023-08-06 Sun>
#+begin_src sh
 nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_NA12878-63118093_S260-GRCh38/callVariant/haplotypecaller/2300346867_NA12878-63118093_S260-GRCh38.vcf.gz --outdir=out/2300346867_NA12878-63118093_S260-GRCh38/happy-nextera-lifted/ --compare=happy -lib lib --capture=capture/nexterarapidcapture_exome_targetedregions_v1.2-nochrM_lifted.bed  --id=HG001 --genome=GRCh38
#+end_src
**** DONE Tester Agilent SureSelect All Exon V8 (hg38) :giab:
CLOSED: [2023-07-31 Mon 23:09] SCHEDULED: <2023-07-31 Mon>
https://earray.chem.agilent.com/suredesign/index.htm
"Find design"
"Agilent catalog"
Fichiers:
- Regions.bed: Targeted exon intervals, curated and targeted by Agilent Technologies
- MergedProbes.bed: Merged probes for targeted enrichment of exons described in Regions.bed
- Covered.bed: Merged probes and sequences with 95% homology or above
- Padded.bed: Merged probes and sequences with 95% homology or above extended 50 bp at each side
- AllTracks.bed: Targeted regions and covered tracks
 #+begin_src sh
nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_63118093_NA12878-GRCh38/callVariant/haplotypecaller/2300346867_63118093_NA12878-GRCh38.vcf.gz --outdir=out/2300346867_63118093_NA12878-GRCh38/happy/ --compare=happy -lib lib --capture=capture/Agilent_SureSelect_All_Exons_v8_hg38_Regions.bed  --id=HG001 --genome=GRCh38
 #+end_src
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision | METRIC.Frac_NA | METRIC.F1_Score | TRUTH.TOTAL.TiTv_ratio | QUERY.TOTAL.TiTv_ratio | TRUTH.TOTAL.het_hom_ratio | QUERY.TOTAL.het_hom_ratio |
| INDEL | ALL    |         423 |      395 |       28 |         915 |      108 |       405 |     4 |    13 |      0.933806 |         0.788235 |       0.442623 |        0.854868 |                        |                        |        1.7012987012987013 |        2.7916666666666665 |
| INDEL | PASS   |         423 |      395 |       28 |         915 |      108 |       405 |     4 |    13 |      0.933806 |         0.788235 |       0.442623 |        0.854868 |                        |                        |        1.7012987012987013 |        2.7916666666666665 |
| SNP   | ALL    |       20984 |    20600 |      384 |       26080 |      780 |      4703 |    62 |    10 |        0.9817 |         0.963512 |        0.18033 |        0.972521 |     3.0499710592321048 |     2.7596541786743516 |          1.58256372367935 |        1.8978207694018234 |
| SNP   | PASS   |       20984 |    20600 |      384 |       26080 |      780 |      4703 |    62 |    10 |        0.9817 |         0.963512 |        0.18033 |        0.972521 |     3.0499710592321048 |     2.7596541786743516 |          1.58256372367935 |        1.8978207694018234 |
**** DONE Test Twist Human core Exome (hg38):giab:
CLOSED: [2023-08-01 Tue 23:16] SCHEDULED: <202 3-08-02 Wed>
https://www.twistbioscience.com/resources/data-files/ngs-human-core-exome-panel-bed-file
#+begin_src
nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_63118093_NA12878-GRCh38/callVariant/haplotypecaller/2300346867_63118093_NA12878-GRCh38.vcf.gz --outdir=out/2300346867_63118093_NA12878-GRCh38/happy-twist-exome-core/ --compare=happy -lib lib --capture=capture/Twist_Exome_Core_Covered_Targets_hg38.bed  --id=HG001 --genome=GRCh38 -bg
#+end_src
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision | METRIC.Frac_NA | METRIC.F1_Score | TRUTH.TOTAL.TiTv_ratio | QUERY.TOTAL.TiTv_ratio | TRUTH.TOTAL.het_hom_ratio | QUERY.TOTAL.het_hom_ratio |
| INDEL | ALL    |         328 |      313 |       15 |         722 |       95 |       309 |     4 |    13 |      0.954268 |         0.769976 |       0.427978 |        0.852273 |                        |                        |        1.8584070796460177 |        2.8967391304347827 |
| INDEL | PASS   |         328 |      313 |       15 |         722 |       95 |       309 |     4 |    13 |      0.954268 |         0.769976 |       0.427978 |        0.852273 |                        |                        |        1.8584070796460177 |        2.8967391304347827 |
| SNP   | ALL    |       19198 |    18962 |      236 |       23381 |      684 |      3738 |    48 |    10 |      0.987707 |         0.965178 |       0.159873 |        0.976313 |     3.1034188034188035 |      2.859264147830391 |        1.5669565217391304 |        1.8578767123287672 |
| SNP   | PASS   |       19198 |    18962 |      236 |       23381 |      684 |      3738 |    48 |    10 |      0.987707 |         0.965178 |       0.159873 |        0.976313 |     3.1034188034188035 |      2.859264147830391 |        1.5669565217391304 |        1.8578767123287672 |
**** DONE Test Twist Human core Exome (hg38):giab:
CLOSED: [2023-08-05 Sat 09:25] SCHEDULED: <2023-08-03 Thu 20:00>
#+begin_src sh
ID="2300346867_NA12878-63118093_S260-GRCh38"; nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/${ID}/callVariant/haplotypecaller/${ID}.vcf.gz --outdir=out/${ID}/happy-twist-exome-core/ --compare=happy -lib lib --capture=capture/Twist_Exome_Core_Covered_Targets_hg38.bed  --id=HG001 --genome=GRCh38 -bg
#+end_src
**** DONE Tester Agilen SureSelect All Exon V8 (hg38) GATK-4.4:giab:
CLOSED: [2023-08-05 Sat 09:25] SCHEDULED: <2023-08-03 Thu 20:00>
**** DONE Vérifier l'impact gatk 4.3 - 4.4 : aucun
CLOSED: [2023-08-05 Sat 09:25]
**** DONE Figure comparant les 3 capture :hg001:
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** DONE Mail Paul sur  les 3 capture :hg001:
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** KILL Tester si le panel Twist Alliance VCGS Exome suffit
CLOSED: [2023-07-31 Mon 22:31] SCHEDULED: <2023-07-30 Sun>
**** DONE Mail cento pour demande le type de capture
CLOSED: [2023-10-07 Sat 17:59]
/Entered on/ [2023-08-07 Mon 20:40]
Twist exome
*** PROJ Comparer happy et happy-vcfeval :giab:
** TODO Données syndip (CHM-eval) : non car génome ! :syndip:
https://github.com/lh3/CHM-eval
*** KILL Données officielles : non car génome !!
CLOSED: [2023-11-19 Sun 23:43]
**** KILL Run ERR1341793
CLOSED: [2023-11-19 Sun 23:43] SCHEDULED: <2023-11-18 Sat>
(raw reads ERR1341793_1.fastq.gz and ERR1341793_2.fastq.gz downloaded from https://www.ebi.ac.uk/ena/browser/view/ERR1341793)
**** KILL Run ERR1341796
CLOSED: [2023-11-19 Sun 23:43] SCHEDULED: <2023-11-18 Sat>
*** TODO Donn
ées exome Broad institute (nextflow)
https://console.cloud.
google.com/storage/browser/broad-public-datasets/CHM1_CHM13_WES;tab=objects?pli=1&prefix=&forceOnObjectsSortingFiltering=false
*** TODO Télécharger VCF avec nextflow
SCHEDULED: <2023-11-25 Sat>
https://github.com/lh3/CHM-eval/releases
** TODO Insilico :cento:
*** TODO tous les variants centogène
**** DONE Extraire liste des SNVs
CLOSED: [2023-04-22 Sat 17:32] SCHEDULED: <2023-04-17 Mon>
***** DONE Corriger manquant à la main
CLOSED: [2023-04-22 Sat 17:31]
La sortie est sauvegardé dans git-annex : variants_success.csv
***** DONE Automatique
CLOSED: [2023-04-22 Sat 17:31]
**** DONE Convert SNVs : transcript -> génomique
CLOSED: [2023-06-03 Sat 17:16]
***** DONE Variant_recoder
CLOSED: [2023-04-26 Wed 21:21] SCHEDULED: <2023-04-22 Sat>
****** KILL Haskell: 160 manquant : recoded-success.csv
CLOSED: [2023-04-25 Tue 18:32]
La liste des variants a été générée en Haskel   l et nettoyée à la main.
On générer une liste de variant pour variant_rec            oder et on soumet tout d'un coup.
[[file:~/recherche/bisonex/parsevariants/app/Main.hs][parsevariant]]
#+begin_src haskell
recodeVariant = do
  prepareVariantRecod   er "variant_success.csv" "renamed.csv"
  runVariantRecoder "renamed.csv" "recoded.json"
#+end_src
#+RESULTS:
: <interactive>:4:3-19: error:
:     Variable not in scope: runVariantRecoder :: String -> String -> t
: gh
Problème : 160 n'ont pas pu être lu sur 820, probablement à cause du numéro mineur de transcrit
La sortie est sauvegardé dans git-annex : variants-recoded-raw.json.
****** KILL Julia
CLOSED: [2023-04-25 Tue 18:32]
On regénère la liste de variant et on passe à Julia pour préparer l'appel en parallèle à variant recoder
[[file:~/recherche/bisonex/parsevariants/variantRecoder.jl][variantRecoder.jl]]
#+begin_src julia
setupVariantRecoder(unique(init), n)
#+end_src
Puis
#+begin_src sh
parallel -a parallel-recoder.sh --jobs 10
#+end_src
On récupère les résultats
#+begin_src julia
(fails, success) = mergeVariantRecoder(n)
CSV.write(fSuccess, success)
CSV.write(fFailures, fails)
#+end_src
Certains variants ne sont pas trouvé, donc on prépare un nouveau job en enlevant les versionrs mineures des transcrits
#+begin_src julia
# Cleanup json and txt
if isfile(fSuccess) && isfile(fFailures)
    foreach(rm, variantRecoderInput())
    foreach(rm, variantRecoderOutput())
end
redoFails(fFailures)
#+end_src
Puis
#+begin_src sh
parallel -a parallel-recoder.sh --jobs 3
#+end_src
Il manque encore 70 transcrits
***** DONE Julia avec mobidetails: recode-failures-mobidetails.csv
CLOSED: [2023-04-25 Tue 18:58]
Nouvelle stratégie : on essaie une fois variant recoder.
Pour tous les échecs, on utilise mobidetails (~170).
Si l'ID n'est pas trouvé, on incrémente le numéro de version 2 fois
***** DONE Reste une dizaine à corriger à la main
CLOSED: [2023-04-26 Wed 21:21]
- [X] certains transcrits ont juste été supprimé
- [X] Erreur de parsing, manque souvent un -
#+begin_src julia
lastTryMobidetails("recoded-failures-mobidetails.csv")
#+end_src
***** DONE Fusionner données
CLOSED: [2023-04-26 Wed 22:35]
#+begin_src julia
function mergeAllGenomic()
    dNew = mergeAll("recoded-success.csv",
                    "recoded-failures-mobidetails.csv",
                    "recoded-failures-mobidetails-redo.csv")
    dInit = @chain DataFrame(CSV.File("variant_success.csv")) begin
        @transform :transcript = :transcript .* ":" .* :coding .* :codingPos .* :codingChange
        @select :file :transcript :classification :zygosity
        @rename :classificationCento = :classification
    end
    dTmp = outerjoin(dInit, dNew, on = :transcript)
    CSV.write("variant_genomic.csv", dTmp)
end
fSuccess = "recoded-success.csv"
fFailures = "recoded-failures.csv"
# variantRecoder(fSuccess, fFailures)
# mobidetailsOnFailures(fFailures)
# lastTryMobidetails("recoded-failures-mobidetails.csv")
mergeAllGenomic()
#+end_src
***** DONE Formatter donner pour simuscop
CLOSED: [2023-04-28 Fri 11:55] SCHEDULED: <2023-04-26 Wed>
**** KILL Extraire liste des CNVs
CLOSED: [2023-08-12 Sat 15:54]
SCHEDULED: <2023-04-17 Mon>
**** KILL Simuscop :simuscop:
CLOSED: [2023-08-12 Sat 15:54]
***** DONE Entrainer le modèle sur 63003856/
CLOSED: [2023-04-29 Sat 19:56]
Relancer le modèle pour être sûr
***** DONE Générer fastq avec simuscop (del et ins seulement) 20x
CLOSED: [2023-04-28 Fri 23:35] SCHEDULED: <2023-04-22 Sat>
****** DONE Génerer un profile avec bed de centogène
CLOSED: [2023-04-28 Fri 11:54] SCHEDULED: <2023-04-22 Sat>
NA12878 mais à refaire avec un vrai séquencage
Voir [[*Centogène][Bed Centogène]] pour choix
****** DONE Générer les données en 20x
CLOSED: [2023-04-28 Fri 11:54] SCHEDULED: <2023-04-22 Sat>
capture de cento
****** DONE Regénérer en supprimant les doublons
CLOSED: [2023-04-28 Fri 17:28]
***** DONE Quelle couverture ?
CLOSED: [2023-04-29 Sat 18:26]
ex sur chr11:16,014,966 où on a 11 reads dans la simulation contre 200 !
****** 200 est la plus proche
#+attr_html: :width 500px
[[./simuscop-200-chr1-1.png]]
#+attr_html: :width 500px
[[./simuscop-200-chr1-2.png]]
****** DONE 20x
CLOSED: [2023-04-29 Sat 15:38]
****** DONE 50x
CLOSED: [2023-04-29 Sat 15:38]
****** DONE 100x
CLOSED: [2023-04-29 Sat 15:39]
****** DONE 200x
CLOSED: [2023-04-29 Sat 15:39]
***** DONE Reads mal centrés sur des petits exons seuls
CLOSED: [2023-04-29 Sat 19:56] SCHEDULED: <2023-04-29 Sat>
Capture ok : [[https://genome-euro.ucsc.edu/cgi-bin/hgTracks?db=hg38&lastVirtModeType=default&lastVirtModeExtraState=&virtModeType=default&virtMode=0&nonVirtPosition=&position=chr1%3A153817168%2D153817824&hgsid=296556270_F4fkENLPXHXidi2oALXls2jxNH9l][UCSC]] (track noire)
Mais mauvaise répartitiopn
#+attr_html: :width 800px
[[./simuscop-error.png]]
À tester
- Problème de profile ?
  - mauvais patient ?
  - mauvaise génération ? -> comparer avec ceux donnés sur github
- nom des chromosomes ?
****** DONE [#A] Tester sur exon 6 GATAD2B pour NC_000001.11:g.153817496A>T
CLOSED: [2023-04-29 Sat 19:56] SCHEDULED: <2023-04-29 Sat>
******* DONE Configuration + Profile 63003856.profile: idem, mal centré
CLOSED: [2023-04-29 Sat 19:18]
Téléchargement des données
#+begin_src sh :dir ~/code/bisonex/test-simuscop
scp meso:/Work/Projects/bisonex/data/genome/GRCh38.p14/genomeRef.fna .
scp meso:Work/Projects/bisonex/data/simuscop/*.profile .
scp -r meso:/Work/Projects/bisonex/data/genome/GRCh38.p13/bwa .
#+end_src
On récupère l'exon (NB: org-mode ne lance pas le code...)
#+begin_src julia
using CSV,DataFramesMeta
d = CSV.read("VCGS_Exome_Covered_Targets_hg38_40.1MB_renamed.bed", header=false, delim="\t", DataFrame)
@subset d :Column1 .== "NC_000001.11" :Column2 .<= 153817496 :Column3 .>= 153817496
#+end_src
NC_000001.11  153817371  153817542
Génération du bed
#+begin_src sh :dir ~/code/bisonex/test-simuscop
echo -e "NC_000001.11\t153817371\t153817542" > gatad2b-exon6.bed
#+end_src
#+RESULTS:
Génération d'un variant
#+begin_src sh :dir ~/code/bisonex/test-simuscop
echo -e "s\tsingle\tNC_000001.11\t153817496\tA\tT\thet"> variant.txt
#+end_src
#+RESULTS:
Génération du fichier de config
#+begin_src sh :dir ~/code/bisonex/test-simuscop
cat > config_wes.txt << EOL
ref = genomeRef.fna
profile = ./63003856.profile
variation = ./variant.txt
target = ./gatad2b-exon6.bed
layout = PE
threads = 1
name = single
output = test-gatad2b
coverage = 20
EOL
#+end_src
#+RESULTS:
On démarre la simulation
#+begin_src sh :dir ~/code/bisonex/test-simuscop
simuReads config_wes.txt
#+end_src
#+RESULTS:
Alignement
#+begin_src sh :dir ~/code/bisonex/test-simuscop
bwa mem -R '@RG\tID:sample\tSM:sample\tPL:ILLUMINA\tPM:Miseq\tCN:lol\tLB:definition_to_add' bwa/genomeRef test-gatad2b/single_1.fq  test-gatad2b/single_2.fq | samtools sort  -o single.bam
#+end_src
#+RESULTS:
******* DONE Profile github  HiSeq2000
CLOSED: [2023-04-29 Sat 19:56]
#+begin_src sh :dir ~/code/bisonex/test-simuscop :result file
wget https://raw.githubuserco

[18.49719]

[21.93308]

    0.529245 |
Hg38
| Type  | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision |
| INDEL |         549 |      489 |       60 |         899 |       64 |       340 |     8 |    17 |      0.890710 |         0.885510 |
| SNP   |       21973 |    21462 |      511 |       26285 |      563 |      4263 |    68 |    16 |      0.976744 |         0.974435 |
****** DONE Interesection des bed: similaire
CLOSED: [2023-07-04 Tue 23:11]
HG38
 #+begin_src sh
 bedtools intersect -a capture/Agilent_SureSelect_All_Exons_v7_hg38_Regions.bed -b /Work/Groups/bisonex/data/giab/GRCh38/HG001_GRCh38_1_22_v4.2.1_benchmark.bed  | wc -l
 #+end_src
 204280
 T2T
 #+begin_src sh
 bedtools intersect -a /Work/Groups/bisonex/data/giab/T2T/Agilent_SureSelect_All_Exons_v7_hg38_Regions_hg38_T2T.bed -b /Work/Groups/bisonex/data/giab/T2T/HG001_GRCh38_1_22_v4.2.1_benchmark_hg38_T2T.bed  | wc -l
 #+end_src
 204021
****** DONE Vérifier la ligne de commande
CLOSED: [2023-07-04 Tue 23:38]
#+begin_src sh
hap.py \
    HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz \
    HG001-SRX11061486_SRR14724513-T2T.vcf.gz \
     \
    --reference chm13v2.0.fa \
    --threads 6 \
     \
    -T Agilent_SureSelect_All_Exons_v7_hg38_Regions_hg38_T2T.bed \
    --false-positives HG001_GRCh38_1_22_v4.2.1_benchmark_hg38_T2T.bed \
     \
    -o HG001
#+end_src
****** DONE Corriger FILTER : mieux mais toujours trop de négatifs. 3/4 SNP retrouvés
CLOSED: [2023-07-08 Sat 15:19] SCHEDULED: <2023-07-08 Sat>
 Type Filter  TRUTH.TOTAL  TRUTH.TP  TRUTH.FN  QUERY.TOTAL  QUERY.FP  QUERY.UNK  FP.gt  FP.al  METRIC.Recall  METRIC.Precision  METRIC.Frac_NA  METRIC.F1_Score  TRUTH.TOTAL.TiTv_ratio  QUERY.TOTAL.TiTv_ratio  TRUTH.TOTAL.het_hom_ratio  QUERY.TOTAL.het_hom_ratio
INDEL    ALL          413       246       167          751       289        215      2     98       0.595642          0.460821        0.286285         0.519629                     NaN                     NaN                   2.428571                   2.465116
INDEL   PASS          413       246       167          751       289        215      2     98       0.595642          0.460821        0.286285         0.519629                     NaN                     NaN                   2.428571                   2.465116
  SNP    ALL        15883     15479       404        23597      5277       2841     46     44       0.974564          0.745760        0.120397         0.844947                3.017198                 2.85705                   5.560099                   2.114633
  SNP   PASS        15883     15479       404        23597      5277       2841     46     44       0.974564          0.745760        0.120397         0.844947                3.017198                 2.85705                   5.560099                   2.114633
******* DONE Vérifier qu'il ne reste plus de filtre autre que PASS
CLOSED: [2023-07-08 Sat 15:19]
#+begin_src
$ zgrep -c 'PASS' HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz
3730505
$ zgrep -c '^chr' HG001_GRCh38_1_22_v4_lifted_merged.vcf.gz
3730506
#+end_src
****** TODO 1/4 SNP manquant ?
******* DONE Regarder avec Julia si ce sont vraiment des FP: 61/5277 qui ne le sont pas
CLOSED: [2023-07-09 Sun 12:09]
******* DONE Examiner les FP
CLOSED: [2023-07-30 Sun 22:05]
******* DONE Tester un FP
CLOSED: [2023-07-30 Sun 22:05]
  2 │ chr1        608765  A           G           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:ti:SNP:homalt:188
  liftDown UCSC: rien en GIAB : vrai FP
 3 │ chr1        762943  A           G           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:ti:SNP:homalt:287
 4 │ chr1        762945  A           T           ./.:.:.:.:NOCALL:nocall:.  1/1:FP:.:tv:SNP:homalt:287
 Remaniements complexes ? Pas dans le gène en HG38
******* DONE La plupart des FP (4705/5566) sont homozygotes: erreur de référence ?
CLOSED: [2023-07-12 Wed 21:10] SCHEDULED: <2023-07-09 Sun>
Sur les 2 premiers variants, ils montrent en fait la différence entre T2T et GRCh38
Erreur à l'alignement ?
******** KILL relancer l'alignement
CLOSED: [2023-07-09 Sun 17:36]
******** DONE vérifier reads identiques hg38 et T2T: oui
CLOSED: [2023-07-09 Sun 16:36]
T2T CHR1608765
38   	chr1:1180168-1180168 (
SRR14724513.24448214
SRR14724513.24448214
******* DONE Vérifier quelques variants sur IGV
CLOSED: [2023-07-09 Sun 17:36]
******* KILL Répartition des FP : cluster ?
CLOSED: [2023-07-09 Sun 17:36]
****** DONE Examiner les FP restant après correction selon séquence de référence
CLOSED: [2023-08-12 Sat 15:57]
****** HOLD Examiner les variants supprimé
****** TODO Enlever les FP qui correspondent à un changement dans le génome
******* Condition:
- pas de variation à la position en GRCh38
- variantion homozygote
- la varation en T2T correspond au changement de pair de base GRC38 -> T2T
  pour les SNP:
  alt_T2T[i] = DNA_GRC38[j]
  avec i la position en T2T et j la position en GRCh38
  Note: définir un ID n'est pas correct car les variants peuvent être modifié par happy !
******* Idée
 - Pour chaque FP, c'est un "faux" FP si
     - REF en hg38 == ALT en T2T
     - et REF en hg38 != REF en T2T
     - et variant homozygote
Comment obtenir les séquences de réferences ?
1. liftover
2. blat sur la séquence autour du variant
3. identifier quelques reads contenant le variant et regarder leur aligneement en hg38
Après discussion avec Alexis: solution 3
******* Algorithme
1. Extraire les coordonnées en T2T des faux positifs *homozygote*
2. Pour chaque faux positif
   1. lister 10 reads contenant le variant
   2. pour chacun de ces reads, récupérer la séquence en T2T et GRCh38 via le nom du read dans le bam
   3. si la séquence en T2T modifiée par le variant est "identique" à celle en GRCh38, alors on ignore ce faux positif
Note: on ignore les reads qui ont changé de chromosome entre les version
******* DONE Résultat préliminaire
CLOSED: [2023-07-23 Sun 14:30]
cf [[file:~/roam/research/bisonex/code/giab/giab-corrected.csv][script julia]]
3498 faux positifs en moins, soit 0.89 sensibilité
julia> tp=15479
julia> fp=5277
julia> tp/(tp+fp)
0.7457602620928888
julia> tp/(tp+(fp-3498))
0.8969173716537258
On est toujours en dessous des 97%
******* HOLD Corriger proprement VCF ou résultats Happy
******* TODO Adapter pour gérer plusieurs variants par read
****** DONE Méthodologie du pangenome
CLOSED: [2023-10-03 Tue 21:28]
Voir biblio[cite:@liao2023]  mais ont aligné sur GRCH38
******* DONE Mail alexis
CLOSED: [2023-10-03 Tue 21:28]
****** DONE Méthodologie T2T
CLOSED: [2023-10-16 Mon 19:42]
Mail alexis
SCHEDULED: <2023-10-04 Wed>
***** TODO Rendre simplement le nombre de vrais positifs
SCHEDULED: <2023-12-02 Sat>
***** KILL Mail Yannis
CLOSED: [2023-07-08 Sat 10:44]
***** DONE Mail GIAB pour version T2T
CLOSED: [2023-07-07 Fri 18:37]
**** TODO HG002 :hg002:T2T:
**** TODO HG003 :hg003:T2T:
**** TODO HG004 :hg004:T2T:
**** DONE Plot : ashkenazim trio :hg38:
CLOSED: [2023-07-30 Sun 16:49] SCHEDULED: <2023-07-30 Sun 15:00>
:LOGBOOK:
CLOCK: [2023-07-30 Sun 16:06]--[2023-07-30 Sun 16:35] =>  0:29
CLOCK: [2023-07-30 Sun 15:39]--[2023-07-30 Sun 15:40] =>  0:01
:END:
/Entered on/ [2023-04-16 Sun 17:29]
Refaire résultats
**** DONE Mail Paul sur les résultat ashkenazim +/- centogene
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** DONE Relancer comparaison GIAB avec GATK 4.4.0
CLOSED: [2023-08-12 Sat 15:55]
/Entered on/ [2023-08-03 Thu 12:42]
*** TODO Platinum genome :platinum:
https://emea.illumina.com/platinumgenomes.html
**** TODO Tester sur la zone couverte par l'exome centogène
SCHEDULED: <2023-12-02 Sat>
*** DONE Séquencer NA12878 :cento:hg001:
CLOSED: [2023-10-07 Sat 17:59]
Discussion avec Paul : sous-traitant ne nous donnera pas les données, il faut commander l'ADN
**** DONE ADN commandé
CLOSED: [2023-06-30 Fri 22:29]
**** DONE Sauvegarder les données brutes
CLOSED: [2023-07-30 Sun 14:22] SCHEDULED: <2023-07-19 Wed>
K, scality, S
**** KILL Récupérer le fichier de capture
CLOSED: [2023-07-30 Sun 14:25] SCHEDULED: <2023-07-23 Sun>
Candidats donnés dans publication https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8354858/
#+begin_quote
In short, the Nextera Rapid Capture Exome Kit (Illumina, San Diego, CA), the SureSelect Human All Exon kit (Agilent, Santa Clara, CA) or the Twist Human Core Exome was used for enrichment, and a Nextseq500, HiSeq4000, or Novoseq 6000 (Illumina) instrument was used for the actual sequencing, with the average coverage targeted to at least 100× or at least 98% of the target DNA covered 20×.
#+end_quote
Par défaut, on utilisera https://www.twistbioscience.com/products/ngs/alliance-panels#tab-3
ANnonce récente pour nouveau panel Twist : https://www.centogene.com/news-events/news/newsdetails/twist-bioscience-and-centogene-launch-three-panels-to-advance-rare-disease-and-hereditary-cancer-research-and-support-diagnostics
Masi pas de fichier BED
***** DONE Mail centogène
CLOSED: [2023-07-30 Sun 14:22] DEADLINE: <2023-07-23 Sun>
**** DONE Tester Nextera Rapid Capture Exome v1.2 (hg19) :giab:
CLOSED: [2023-08-06 Sun 19:05] SCHEDULED: <2023-08-03 Thu 19:00>
https://support.illumina.com/downloads/nextera-rapid-capture-exome-v1-2-product-files.html
***** DONE Liftover capture
CLOSED: [2023-08-06 Sun 18:30] SCHEDULED: <2023-08-06 Sun>
#+begin_src sh
 nextflow run -profile standard,helios workflows/lift-nextera-capture.nf  -lib lib
#+end_src
Vérification rapide : ok
***** DONE Run
CLOSED: [2023-08-06 Sun 19:05] SCHEDULED: <2023-08-06 Sun>
#+begin_src sh
 nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_NA12878-63118093_S260-GRCh38/callVariant/haplotypecaller/2300346867_NA12878-63118093_S260-GRCh38.vcf.gz --outdir=out/2300346867_NA12878-63118093_S260-GRCh38/happy-nextera-lifted/ --compare=happy -lib lib --capture=capture/nexterarapidcapture_exome_targetedregions_v1.2-nochrM_lifted.bed  --id=HG001 --genome=GRCh38
#+end_src
**** DONE Tester Agilent SureSelect All Exon V8 (hg38) :giab:
CLOSED: [2023-07-31 Mon 23:09] SCHEDULED: <2023-07-31 Mon>
https://earray.chem.agilent.com/suredesign/index.htm
"Find design"
"Agilent catalog"
Fichiers:
- Regions.bed: Targeted exon intervals, curated and targeted by Agilent Technologies
- MergedProbes.bed: Merged probes for targeted enrichment of exons described in Regions.bed
- Covered.bed: Merged probes and sequences with 95% homology or above
- Padded.bed: Merged probes and sequences with 95% homology or above extended 50 bp at each side
- AllTracks.bed: Targeted regions and covered tracks
 #+begin_src sh
nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_63118093_NA12878-GRCh38/callVariant/haplotypecaller/2300346867_63118093_NA12878-GRCh38.vcf.gz --outdir=out/2300346867_63118093_NA12878-GRCh38/happy/ --compare=happy -lib lib --capture=capture/Agilent_SureSelect_All_Exons_v8_hg38_Regions.bed  --id=HG001 --genome=GRCh38
 #+end_src
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision | METRIC.Frac_NA | METRIC.F1_Score | TRUTH.TOTAL.TiTv_ratio | QUERY.TOTAL.TiTv_ratio | TRUTH.TOTAL.het_hom_ratio | QUERY.TOTAL.het_hom_ratio |
| INDEL | ALL    |         423 |      395 |       28 |         915 |      108 |       405 |     4 |    13 |      0.933806 |         0.788235 |       0.442623 |        0.854868 |                        |                        |        1.7012987012987013 |        2.7916666666666665 |
| INDEL | PASS   |         423 |      395 |       28 |         915 |      108 |       405 |     4 |    13 |      0.933806 |         0.788235 |       0.442623 |        0.854868 |                        |                        |        1.7012987012987013 |        2.7916666666666665 |
| SNP   | ALL    |       20984 |    20600 |      384 |       26080 |      780 |      4703 |    62 |    10 |        0.9817 |         0.963512 |        0.18033 |        0.972521 |     3.0499710592321048 |     2.7596541786743516 |          1.58256372367935 |        1.8978207694018234 |
| SNP   | PASS   |       20984 |    20600 |      384 |       26080 |      780 |      4703 |    62 |    10 |        0.9817 |         0.963512 |        0.18033 |        0.972521 |     3.0499710592321048 |     2.7596541786743516 |          1.58256372367935 |        1.8978207694018234 |
**** DONE Test Twist Human core Exome (hg38):giab:
CLOSED: [2023-08-01 Tue 23:16] SCHEDULED: <202 3-08-02 Wed>
https://www.twistbioscience.com/resources/data-files/ngs-human-core-exome-panel-bed-file
#+begin_src
nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/2300346867_63118093_NA12878-GRCh38/callVariant/haplotypecaller/2300346867_63118093_NA12878-GRCh38.vcf.gz --outdir=out/2300346867_63118093_NA12878-GRCh38/happy-twist-exome-core/ --compare=happy -lib lib --capture=capture/Twist_Exome_Core_Covered_Targets_hg38.bed  --id=HG001 --genome=GRCh38 -bg
#+end_src
| Type  | Filter | TRUTH.TOTAL | TRUTH.TP | TRUTH.FN | QUERY.TOTAL | QUERY.FP | QUERY.UNK | FP.gt | FP.al | METRIC.Recall | METRIC.Precision | METRIC.Frac_NA | METRIC.F1_Score | TRUTH.TOTAL.TiTv_ratio | QUERY.TOTAL.TiTv_ratio | TRUTH.TOTAL.het_hom_ratio | QUERY.TOTAL.het_hom_ratio |
| INDEL | ALL    |         328 |      313 |       15 |         722 |       95 |       309 |     4 |    13 |      0.954268 |         0.769976 |       0.427978 |        0.852273 |                        |                        |        1.8584070796460177 |        2.8967391304347827 |
| INDEL | PASS   |         328 |      313 |       15 |         722 |       95 |       309 |     4 |    13 |      0.954268 |         0.769976 |       0.427978 |        0.852273 |                        |                        |        1.8584070796460177 |        2.8967391304347827 |
| SNP   | ALL    |       19198 |    18962 |      236 |       23381 |      684 |      3738 |    48 |    10 |      0.987707 |         0.965178 |       0.159873 |        0.976313 |     3.1034188034188035 |      2.859264147830391 |        1.5669565217391304 |        1.8578767123287672 |
| SNP   | PASS   |       19198 |    18962 |      236 |       23381 |      684 |      3738 |    48 |    10 |      0.987707 |         0.965178 |       0.159873 |        0.976313 |     3.1034188034188035 |      2.859264147830391 |        1.5669565217391304 |        1.8578767123287672 |
**** DONE Test Twist Human core Exome (hg38):giab:
CLOSED: [2023-08-05 Sat 09:25] SCHEDULED: <2023-08-03 Thu 20:00>
#+begin_src sh
ID="2300346867_NA12878-63118093_S260-GRCh38"; nextflow run workflows/compareVCF.nf -profile standard,helios --query=out/${ID}/callVariant/haplotypecaller/${ID}.vcf.gz --outdir=out/${ID}/happy-twist-exome-core/ --compare=happy -lib lib --capture=capture/Twist_Exome_Core_Covered_Targets_hg38.bed  --id=HG001 --genome=GRCh38 -bg
#+end_src
**** DONE Tester Agilen SureSelect All Exon V8 (hg38) GATK-4.4:giab:
CLOSED: [2023-08-05 Sat 09:25] SCHEDULED: <2023-08-03 Thu 20:00>
**** DONE Vérifier l'impact gatk 4.3 - 4.4 : aucun
CLOSED: [2023-08-05 Sat 09:25]
**** DONE Figure comparant les 3 capture :hg001:
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** DONE Mail Paul sur  les 3 capture :hg001:
CLOSED: [2023-08-06 Sun 20:24] SCHEDULED: <2023-08-06 Sun>
**** KILL Tester si le panel Twist Alliance VCGS Exome suffit
CLOSED: [2023-07-31 Mon 22:31] SCHEDULED: <2023-07-30 Sun>
**** DONE Mail cento pour demande le type de capture
CLOSED: [2023-10-07 Sat 17:59]
/Entered on/ [2023-08-07 Mon 20:40]
Twist exome
*** PROJ Comparer happy et happy-vcfeval :giab:
** TODO Données syndip (CHM-eval) : non car génome ! :syndip:
https://github.com/lh3/CHM-eval
*** KILL Données officielles : non car génome !!
CLOSED: [2023-11-19 Sun 23:43]
**** KILL Run ERR1341793
CLOSED: [2023-11-19 Sun 23:43] SCHEDULED: <2023-11-18 Sat>
(raw reads ERR1341793_1.fastq.gz and ERR1341793_2.fastq.gz downloaded from https://www.ebi.ac.uk/ena/browser/view/ERR1341793)
**** KILL Run ERR1341796
CLOSED: [2023-11-19 Sun 23:43] SCHEDULED: <2023-11-18 Sat>
*** TODO Données exome Broad institute (nextflow)
SCHEDULED: <2023-11-25 Sat 21:00>
https://console.cloud.google.com/storage/browser/broad-public-datasets/CHM1_CHM13_WES;tab=objects?pli=1&prefix=&forceOnObjectsSortingFiltering=false
*** TODO Télécharger VCF
SCHEDULED: <2023-11-26 Sun>
https://github.com/lh3/CHM-eval/releases
** TODO Insilico :cento:
*** TODO tous les variants centogène
**** DONE Extraire liste des SNVs
CLOSED: [2023-04-22 Sat 17:32] SCHEDULED: <2023-04-17 Mon>
***** DONE Corriger manquant à la main
CLOSED: [2023-04-22 Sat 17:31]
La sortie est sauvegardé dans git-annex : variants_success.csv
***** DONE Automatique
CLOSED: [2023-04-22 Sat 17:31]
**** DONE Convert SNVs : transcript -> génomique
CLOSED: [2023-06-03 Sat 17:16]
***** DONE Variant_recoder
CLOSED: [2023-04-26 Wed 21:21] SCHEDULED: <2023-04-22 Sat>
****** KILL Haskell: 160 manquant : recoded-success.csv
CLOSED: [2023-04-25 Tue 18:32]
La liste des variants a été générée en Haskel   l et nettoyée à la main.
On générer une liste de variant pour variant_rec            oder et on soumet tout d'un coup.
[[file:~/recherche/bisonex/parsevariants/app/Main.hs][parsevariant]]
#+begin_src haskell
recodeVariant = do
  prepareVariantRecod   er "variant_success.csv" "renamed.csv"
  runVariantRecoder "renamed.csv" "recoded.json"
#+end_src
#+RESULTS:
: <interactive>:4:3-19: error:
:     Variable not in scope: runVariantRecoder :: String -> String -> t
: gh
Problème : 160 n'ont pas pu être lu sur 820, probablement à cause du numéro mineur de transcrit
La sortie est sauvegardé dans git-annex : variants-recoded-raw.json.
****** KILL Julia
CLOSED: [2023-04-25 Tue 18:32]
On regénère la liste de variant et on passe à Julia pour préparer l'appel en parallèle à variant recoder
[[file:~/recherche/bisonex/parsevariants/variantRecoder.jl][variantRecoder.jl]]
#+begin_src julia
setupVariantRecoder(unique(init), n)
#+end_src
Puis
#+begin_src sh
parallel -a parallel-recoder.sh --jobs 10
#+end_src
On récupère les résultats
#+begin_src julia
(fails, success) = mergeVariantRecoder(n)
CSV.write(fSuccess, success)
CSV.write(fFailures, fails)
#+end_src
Certains variants ne sont pas trouvé, donc on prépare un nouveau job en enlevant les versionrs mineures des transcrits
#+begin_src julia
# Cleanup json and txt
if isfile(fSuccess) && isfile(fFailures)
    foreach(rm, variantRecoderInput())
    foreach(rm, variantRecoderOutput())
end
redoFails(fFailures)
#+end_src
Puis
#+begin_src sh
parallel -a parallel-recoder.sh --jobs 3
#+end_src
Il manque encore 70 transcrits
***** DONE Julia avec mobidetails: recode-failures-mobidetails.csv
CLOSED: [2023-04-25 Tue 18:58]
Nouvelle stratégie : on essaie une fois variant recoder.
Pour tous les échecs, on utilise mobidetails (~170).
Si l'ID n'est pas trouvé, on incrémente le numéro de version 2 fois
***** DONE Reste une dizaine à corriger à la main
CLOSED: [2023-04-26 Wed 21:21]
- [X] certains transcrits ont juste été supprimé
- [X] Erreur de parsing, manque souvent un -
#+begin_src julia
lastTryMobidetails("recoded-failures-mobidetails.csv")
#+end_src
***** DONE Fusionner données
CLOSED: [2023-04-26 Wed 22:35]
#+begin_src julia
function mergeAllGenomic()
    dNew = mergeAll("recoded-success.csv",
                    "recoded-failures-mobidetails.csv",
                    "recoded-failures-mobidetails-redo.csv")
    dInit = @chain DataFrame(CSV.File("variant_success.csv")) begin
        @transform :transcript = :transcript .* ":" .* :coding .* :codingPos .* :codingChange
        @select :file :transcript :classification :zygosity
        @rename :classificationCento = :classification
    end
    dTmp = outerjoin(dInit, dNew, on = :transcript)
    CSV.write("variant_genomic.csv", dTmp)
end
fSuccess = "recoded-success.csv"
fFailures = "recoded-failures.csv"
# variantRecoder(fSuccess, fFailures)
# mobidetailsOnFailures(fFailures)
# lastTryMobidetails("recoded-failures-mobidetails.csv")
mergeAllGenomic()
#+end_src
***** DONE Formatter donner pour simuscop
CLOSED: [2023-04-28 Fri 11:55] SCHEDULED: <2023-04-26 Wed>
**** KILL Extraire liste des CNVs
CLOSED: [2023-08-12 Sat 15:54]
SCHEDULED: <2023-04-17 Mon>
**** KILL Simuscop :simuscop:
CLOSED: [2023-08-12 Sat 15:54]
***** DONE Entrainer le modèle sur 63003856/
CLOSED: [2023-04-29 Sat 19:56]
Relancer le modèle pour être sûr
***** DONE Générer fastq avec simuscop (del et ins seulement) 20x
CLOSED: [2023-04-28 Fri 23:35] SCHEDULED: <2023-04-22 Sat>
****** DONE Génerer un profile avec bed de centogène
CLOSED: [2023-04-28 Fri 11:54] SCHEDULED: <2023-04-22 Sat>
NA12878 mais à refaire avec un vrai séquencage
Voir [[*Centogène][Bed Centogène]] pour choix
****** DONE Générer les données en 20x
CLOSED: [2023-04-28 Fri 11:54] SCHEDULED: <2023-04-22 Sat>
capture de cento
****** DONE Regénérer en supprimant les doublons
CLOSED: [2023-04-28 Fri 17:28]
***** DONE Quelle couverture ?
CLOSED: [2023-04-29 Sat 18:26]
ex sur chr11:16,014,966 où on a 11 reads dans la simulation contre 200 !
****** 200 est la plus proche
#+attr_html: :width 500px
[[./simuscop-200-chr1-1.png]]
#+attr_html: :width 500px
[[./simuscop-200-chr1-2.png]]
****** DONE 20x
CLOSED: [2023-04-29 Sat 15:38]
****** DONE 50x
CLOSED: [2023-04-29 Sat 15:38]
****** DONE 100x
CLOSED: [2023-04-29 Sat 15:39]
****** DONE 200x
CLOSED: [2023-04-29 Sat 15:39]
***** DONE Reads mal centrés sur des petits exons seuls
CLOSED: [2023-04-29 Sat 19:56] SCHEDULED: <2023-04-29 Sat>
Capture ok : [[https://genome-euro.ucsc.edu/cgi-bin/hgTracks?db=hg38&lastVirtModeType=default&lastVirtModeExtraState=&virtModeType=default&virtMode=0&nonVirtPosition=&position=chr1%3A153817168%2D153817824&hgsid=296556270_F4fkENLPXHXidi2oALXls2jxNH9l][UCSC]] (track noire)
Mais mauvaise répartitiopn
#+attr_html: :width 800px
[[./simuscop-error.png]]
À tester
- Problème de profile ?
  - mauvais patient ?
  - mauvaise génération ? -> comparer avec ceux donnés sur github
- nom des chromosomes ?
****** DONE [#A] Tester sur exon 6 GATAD2B pour NC_000001.11:g.153817496A>T
CLOSED: [2023-04-29 Sat 19:56] SCHEDULED: <2023-04-29 Sat>
******* DONE Configuration + Profile 63003856.profile: idem, mal centré
CLOSED: [2023-04-29 Sat 19:18]
Téléchargement des données
#+begin_src sh :dir ~/code/bisonex/test-simuscop
scp meso:/Work/Projects/bisonex/data/genome/GRCh38.p14/genomeRef.fna .
scp meso:Work/Projects/bisonex/data/simuscop/*.profile .
scp -r meso:/Work/Projects/bisonex/data/genome/GRCh38.p13/bwa .
#+end_src
On récupère l'exon (NB: org-mode ne lance pas le code...)
#+begin_src julia
using CSV,DataFramesMeta
d = CSV.read("VCGS_Exome_Covered_Targets_hg38_40.1MB_renamed.bed", header=false, delim="\t", DataFrame)
@subset d :Column1 .== "NC_000001.11" :Column2 .<= 153817496 :Column3 .>= 153817496
#+end_src
NC_000001.11  153817371  153817542
Génération du bed
#+begin_src sh :dir ~/code/bisonex/test-simuscop
echo -e "NC_000001.11\t153817371\t153817542" > gatad2b-exon6.bed
#+end_src
#+RESULTS:
Génération d'un variant
#+begin_src sh :dir ~/code/bisonex/test-simuscop
echo -e "s\tsingle\tNC_000001.11\t153817496\tA\tT\thet"> variant.txt
#+end_src
#+RESULTS:
Génération du fichier de config
#+begin_src sh :dir ~/code/bisonex/test-simuscop
cat > config_wes.txt << EOL
ref = genomeRef.fna
profile = ./63003856.profile
variation = ./variant.txt
target = ./gatad2b-exon6.bed
layout = PE
threads = 1
name = single
output = test-gatad2b
coverage = 20
EOL
#+end_src
#+RESULTS:
On démarre la simulation
#+begin_src sh :dir ~/code/bisonex/test-simuscop
simuReads config_wes.txt
#+end_src
#+RESULTS:
Alignement
#+begin_src sh :dir ~/code/bisonex/test-simuscop
bwa mem -R '@RG\tID:sample\tSM:sample\tPL:ILLUMINA\tPM:Miseq\tCN:lol\tLB:definition_to_add' bwa/genomeRef test-gatad2b/single_1.fq  test-gatad2b/single_2.fq | samtools sort  -o single.bam
#+end_src
#+RESULTS:
******* DONE Profile github  HiSeq2000
CLOSED: [2023-04-29 Sat 19:56]
#+begin_src sh :dir ~/code/bisonex/test-simuscop :result file
wget https://raw.githubuserco

Replacement in projects/bisonex.org at line 77 [17.35]

B:BD[22.43105] → [13.8613:16805]

:
VEP chooses one block of annotation per variant, using an ordered set of criteria. This order may be customised using --pick_order.
    MANE Select transcript status
    MANE Plus Clinical transcript status
    canonical status of transcript
    APPRIS isoform annotation
    transcript support level
    biotype of transcript ("protein_coding" preferred)
    CCDS status of transcript
    consequence rank according to this table
    translated, transcript or feature length (longer preferred)
"Wherever possible we would discourage you from summarising data in this way. "
**** DONE Mail alexis
CLOSED: [2023-08-20 Sun 13:45] SCHEDULED: <2023-08-20 Sun>
**** TODO Données simuscop 200x
SCHEDULED: <2023-11-26 Sun>
**** DONE En T2T avec liftover (filtre = spip) : ok mais lent et trop de variants :tests:
CLOSED: [2023-09-17 Sun 17:13] SCHEDULED: <2023-09-17 Sun>
1. Conversion en bed
#+begin_src sh :dir:~/code/sanger
open snvs-cento-sanger.csv | select chrom pos | insert pos2 {$in.pos } | to csv --separator="\t" | save snvs-cento-sanger.bed -f
#+end_src
2. Liftover avec UCSC (en ligne)
NB: vérifié sur le premier résultat en cherche le read contenant le variant (samtools view -r puis samtools view | grep en T2T) et avec l'aide d'IGV, on a un variant qui correspond en
chr1:10757746
3. En supposant que l'ordre des variants n'a pas changé, on ajoute simplement REF et ALT avec annotateLifted.jl
Annotation spip *très lente* : 1h13 !
Résultat:
2×3 DataFrame
 Row │ variant              meanQual  depth
     │ String               Float64   Int64
─────┼──────────────────────────────────────
   1 │ chr12:g.13594572      60.0      1
   2 │ chr17:g.10204026      60.0      1
144 found over 146
filter depth : another 0 missed variants
filter poly : another 0 missed variants
filter vep   : another 0 missed variants
Et on a trop de variants en sortie (7330 !)
**** DONE Mail Paul avec résultats filtre en T2T + nouveau schéma
CLOSED: [2023-09-17 Sun 23:15] SCHEDULED: <2023-09-17 Sun>
** TODO Medically relevant genes
SCHEDULED: <2023-11-25 Sat>
/Entered on/ [2023-10-18 Wed 22:37]
* Ré-interprétation :reanalysis:
** DONE Lancer tests sur données brutes [225/250] <(samples.csv)>  <(runs.waiting)>
CLOSED: [2023-10-14 Sat 11:58] SCHEDULED: <2023-10-08 Sun>
- [X] 100222_63015289
- [X] 1600304839_63051311
- [X] 1900007827_62913191
- [X] 1900398899_62999500
- [X] 1900486799_62913197
- [X] 2100422923_62952677
- [X] 2100458888_62933047
- [X] 2100601558_62903840
- [X] 2100609288_62905768
- [X] 2100609501_62905776
- [X] 2100614493_62951074
- [X] 2100622566_62908067
- [X] 2100622601_62908060
- [X] 2100622705_62908063
- [X] 2100640027_62911936
- [X] 2100645285_62913212
- [X] 2100661411_62914081
- [X] 2100661462_62914086
- [X] 2100708257_62921596
- [X] 2100738732_62926501
- [X] 2100738850_62926509
- [X] 2100746751_62926505
- [X] 2100746797_62926506
- [X] 2100782349_62931722
- [X] 2100782416_62931561
- [X] 2100782559_62931718
- [X] 2100799204_62934768
- [X] 2200010202_62940284
- [X] 2200023600_62940631
- [X] 2200024348_62999591
- [X] 2200027505_62942457
- [X] 2200038776_62943412
- [X] 2200041919_62943405
- [X] 2200088014_62951326
- [X] 2200146652_62959388
- [X] 2200151850_62960953
- [X] 2200160014_62959475
- [X] 2200160070_62959478
- [X] 2200201368_62967471
- [X] 2200201400_62967470
- [X] 2200265558_62976332
- [X] 2200265605_62976401
- [X] 2200267046_62975192
- [X] 2200273878_62999530
- [X] 2200279708_62977002
- [X] 2200284408_62979102
- [X] 2200293987_62979116
- [X] 2200294359_62979118
- [X] 2200306299_62982217
- [X] 2200306539_62982193
- [X] 220030671_62982211
- [X] 2200307058_62982231
- [X] 2200307108_62982196
- [X] 2200307136_62982221
- [X] 2200307199_62982239
- [X] 2200307230_62982234
- [X] 2200307262_62982219
- [X] 2200307297_62982227
- [X] 2200324510_62985453
- [X] 2200324549_62985478
- [X] 2200324573_62985445
- [X] 2200324594_62985467
- [X] 2200324606_62985463
- [X] 2200324614_62985459
- [X] 2200338306_62985430
- [X] 2200343880_62989407
- [X] 2200343910_62989460
- [X] 2200343938_62989451
- [X] 2200343966_62989456
- [X] 2200343993_62989440
- [X] 2200344013_62989464
- [X] 2200349749_62989465
- [X] 2200363462_62988848
- [X] 2200377880_62991993
- [X] 2200378032_62991991
- [X] 2200383996_62993828
- [X] 2200384015_62993796
- [X] 2200384046_62993822
- [X] 2200384117_62993808
- [X] 2200384187_62993825
- [X] 2200384231_62992898
- [X] 2200385658_63060260
- [X] 2200394260_62994732
- [X] 2200395817_62994742
- [X] 2200396731_62994737
- [X] 2200424073_62999579
- [X] 2200424207_62999632
- [X] 2200426178_62999630
- [X] 2200426243_62999635
- [X] 2200426466_62999605
- [X] 2200426642_62999627
- [X] 2200427406_62999649
- [X] 2200427512_62999639
- [X] 2200428953_62999572
- [X] 2200428981_62999600
- [X] 2200428999_62999592
- [X] 2200441970_63000868
- [X] 2200441989_63000882
- [X] 2200442135_63000864
- [X] 2200442216_63000886
- [X] 2200442257_63000951
- [X] 2200451801_63003573
- [X] 2200451862_63004218
- [X] 2200451894_63004210
- [X] 2200456165_63051294
- [X] 2200459865_63004933
- [X] 2200459968_63004937
- [X] 2200460073_63004943
- [X] 2200460121_63004684
- [X] 2200467051_63003856
- [X] 2200467225_63004940
- [X] 2200467261_63004930
- [X] 2200467338_63004925
- [X] 2200470099_63004485
- [X] 2200470142_63004480
- [X] 2200471780_63004362
- [X] 2200480910_63006466
- [X] 2200495073_63010427
- [X] 2200495510_63009152
- [X] 2200508677_63060252
- [X] 2200510531_63012582
- [X] 2200510628_63012549
- [X] 2200510657_63012554
- [X] 2200511249_63012533
- [X] 2200511274_63012586
- [X] 2200517952_63060399
- [X] 2200519525_63060439
- [X] 2200524009_63014044
- [X] 2200524609_63014046
- [X] 2200524616_63014048
- [X] 2200533429_63060425
- [X] 2200539735_63060406
- [X] 2200549908_63019339
- [X] 2200549965_63019349
- [X] 2200550414_63019357
- [X] 2200550471_63020031
- [X] 2200550490_63019351
- [X] 2200550505_63019340
- [X] 2200555565_63018614
- [X] 2200559438_63020029
- [X] 2200559682_63020030
- [X] 2200559713_63019623
- [X] 2200559739_63019626
- [X] 2200569969_63019991
- [X] 2200570001_63021580
- [X] 2200570025_63021490
- [X] 2200570035_63021491
- [X] 2200570042_63021493
- [X] 2200570050_63021494
- [X] 2200579897_63024910
- [X] 2200583995_63024866
- [X] 2200584035_63024905
- [X] 2200584069_63024888
- [X] 2200584126_63024810
- [X] 2200589507_63026712
- [X] 2200597365_63027994
- [X] 2200597480_63027988
- [X] 2200597752_63026853
- [X] 2200597778_63027992
- [X] 22005977_63026903
- [X] 2200609031_63026527
- [X] 2200614198_63113928
- [X] 2200620372_63030821
- [X] 2200620442_63030810
- [X] 2200620498_63030816
- [X] 2200620628_63031031
- [X] 2200622310_63030984
- [X] 2200622355_63030956
- [X] 2200625369_63028699
- [X] 2200625410_63028697
- [X] 2200625536_63028694
- [X] 2200630189_63030665
- [X] 2200635149_63033182
- [X] 2200644544_63037731
- [X] 2200644594_63037725
- [X] 2200650089_63038093
- [X] 2200666292_63076568
- [X] 2200669188_63036688
- [X] 2200669320_63040259
- [X] 2200669383_63040254
- [X] 2200669414_63040257
- [X] 2200669446_63040251
- [X] 2200680342_63105271
- [X] 2200694535_63042853
- [X] 2200694789_63042862
- [X] 2200694858_63042702
- [X] 2200694917_63042696
- [X] 2200699290_63043047
- [X] 2200699345_63040238
- [X] 2200699383_63043050
- [X] 2200699412_63040731
- [X] 220071551_63048935
- [X] 2200731515_63048963
- [X] 2200748145_63051198
- [X] 2200748171_63051213
- [X] 2200751046_63051249
- [X] 2200751101_63051234
- [X] 2200766471_63054590
- [X] 2200767731_63054595
- [X] 2200767822_63054464
- [X] 2200775505_63060410
- [X] 2200850441_63019345
- [X] 220597589_63026879
- [X] 2300003253_63060430
- [X] 2300005679_63060370
- [X] 2300009914_63060390
- [X] 2300028784_63060001
- [X] 2300036815_63063357
- [X] 2300055382_63061874
- [X] 2300055421_63061871
- [X] 2300055440_63061880
- [X] 230006894_63064950
- [X] 2300071111_63070356
- [X] 2300083434_

[22.43105]

[15.34952]

:
VEP chooses one block of annotation per variant, using an ordered set of criteria. This order may be customised using --pick_order.
    MANE Select transcript status
    MANE Plus Clinical transcript status
    canonical status of transcript
    APPRIS isoform annotation
    transcript support level
    biotype of transcript ("protein_coding" preferred)
    CCDS status of transcript
    consequence rank according to this table
    translated, transcript or feature length (longer preferred)
"Wherever possible we would discourage you from summarising data in this way. "
**** DONE Mail alexis
CLOSED: [2023-08-20 Sun 13:45] SCHEDULED: <2023-08-20 Sun>
**** TODO Données simuscop 200x
SCHEDULED: <2023-11-26 Sun>
**** DONE En T2T avec liftover (filtre = spip) : ok mais lent et trop de variants :tests:
CLOSED: [2023-09-17 Sun 17:13] SCHEDULED: <2023-09-17 Sun>
1. Conversion en bed
#+begin_src sh :dir:~/code/sanger
open snvs-cento-sanger.csv | select chrom pos | insert pos2 {$in.pos } | to csv --separator="\t" | save snvs-cento-sanger.bed -f
#+end_src
2. Liftover avec UCSC (en ligne)
NB: vérifié sur le premier résultat en cherche le read contenant le variant (samtools view -r puis samtools view | grep en T2T) et avec l'aide d'IGV, on a un variant qui correspond en
chr1:10757746
3. En supposant que l'ordre des variants n'a pas changé, on ajoute simplement REF et ALT avec annotateLifted.jl
Annotation spip *très lente* : 1h13 !
Résultat:
2×3 DataFrame
 Row │ variant              meanQual  depth
     │ String               Float64   Int64
─────┼──────────────────────────────────────
   1 │ chr12:g.13594572      60.0      1
   2 │ chr17:g.10204026      60.0      1
144 found over 146
filter depth : another 0 missed variants
filter poly : another 0 missed variants
filter vep   : another 0 missed variants
Et on a trop de variants en sortie (7330 !)
**** DONE Mail Paul avec résultats filtre en T2T + nouveau schéma
CLOSED: [2023-09-17 Sun 23:15] SCHEDULED: <2023-09-17 Sun>
** TODO Medically relevant genes
SCHEDULED: <2023-11-30 Thu>
/Entered on/ [2023-10-18 Wed 22:37]
* Ré-interprétation :reanalysis:
** DONE Lancer tests sur données brutes [225/250] <(samples.csv)>  <(runs.waiting)>
CLOSED: [2023-10-14 Sat 11:58] SCHEDULED: <2023-10-08 Sun>
- [X] 100222_63015289
- [X] 1600304839_63051311
- [X] 1900007827_62913191
- [X] 1900398899_62999500
- [X] 1900486799_62913197
- [X] 2100422923_62952677
- [X] 2100458888_62933047
- [X] 2100601558_62903840
- [X] 2100609288_62905768
- [X] 2100609501_62905776
- [X] 2100614493_62951074
- [X] 2100622566_62908067
- [X] 2100622601_62908060
- [X] 2100622705_62908063
- [X] 2100640027_62911936
- [X] 2100645285_62913212
- [X] 2100661411_62914081
- [X] 2100661462_62914086
- [X] 2100708257_62921596
- [X] 2100738732_62926501
- [X] 2100738850_62926509
- [X] 2100746751_62926505
- [X] 2100746797_62926506
- [X] 2100782349_62931722
- [X] 2100782416_62931561
- [X] 2100782559_62931718
- [X] 2100799204_62934768
- [X] 2200010202_62940284
- [X] 2200023600_62940631
- [X] 2200024348_62999591
- [X] 2200027505_62942457
- [X] 2200038776_62943412
- [X] 2200041919_62943405
- [X] 2200088014_62951326
- [X] 2200146652_62959388
- [X] 2200151850_62960953
- [X] 2200160014_62959475
- [X] 2200160070_62959478
- [X] 2200201368_62967471
- [X] 2200201400_62967470
- [X] 2200265558_62976332
- [X] 2200265605_62976401
- [X] 2200267046_62975192
- [X] 2200273878_62999530
- [X] 2200279708_62977002
- [X] 2200284408_62979102
- [X] 2200293987_62979116
- [X] 2200294359_62979118
- [X] 2200306299_62982217
- [X] 2200306539_62982193
- [X] 220030671_62982211
- [X] 2200307058_62982231
- [X] 2200307108_62982196
- [X] 2200307136_62982221
- [X] 2200307199_62982239
- [X] 2200307230_62982234
- [X] 2200307262_62982219
- [X] 2200307297_62982227
- [X] 2200324510_62985453
- [X] 2200324549_62985478
- [X] 2200324573_62985445
- [X] 2200324594_62985467
- [X] 2200324606_62985463
- [X] 2200324614_62985459
- [X] 2200338306_62985430
- [X] 2200343880_62989407
- [X] 2200343910_62989460
- [X] 2200343938_62989451
- [X] 2200343966_62989456
- [X] 2200343993_62989440
- [X] 2200344013_62989464
- [X] 2200349749_62989465
- [X] 2200363462_62988848
- [X] 2200377880_62991993
- [X] 2200378032_62991991
- [X] 2200383996_62993828
- [X] 2200384015_62993796
- [X] 2200384046_62993822
- [X] 2200384117_62993808
- [X] 2200384187_62993825
- [X] 2200384231_62992898
- [X] 2200385658_63060260
- [X] 2200394260_62994732
- [X] 2200395817_62994742
- [X] 2200396731_62994737
- [X] 2200424073_62999579
- [X] 2200424207_62999632
- [X] 2200426178_62999630
- [X] 2200426243_62999635
- [X] 2200426466_62999605
- [X] 2200426642_62999627
- [X] 2200427406_62999649
- [X] 2200427512_62999639
- [X] 2200428953_62999572
- [X] 2200428981_62999600
- [X] 2200428999_62999592
- [X] 2200441970_63000868
- [X] 2200441989_63000882
- [X] 2200442135_63000864
- [X] 2200442216_63000886
- [X] 2200442257_63000951
- [X] 2200451801_63003573
- [X] 2200451862_63004218
- [X] 2200451894_63004210
- [X] 2200456165_63051294
- [X] 2200459865_63004933
- [X] 2200459968_63004937
- [X] 2200460073_63004943
- [X] 2200460121_63004684
- [X] 2200467051_63003856
- [X] 2200467225_63004940
- [X] 2200467261_63004930
- [X] 2200467338_63004925
- [X] 2200470099_63004485
- [X] 2200470142_63004480
- [X] 2200471780_63004362
- [X] 2200480910_63006466
- [X] 2200495073_63010427
- [X] 2200495510_63009152
- [X] 2200508677_63060252
- [X] 2200510531_63012582
- [X] 2200510628_63012549
- [X] 2200510657_63012554
- [X] 2200511249_63012533
- [X] 2200511274_63012586
- [X] 2200517952_63060399
- [X] 2200519525_63060439
- [X] 2200524009_63014044
- [X] 2200524609_63014046
- [X] 2200524616_63014048
- [X] 2200533429_63060425
- [X] 2200539735_63060406
- [X] 2200549908_63019339
- [X] 2200549965_63019349
- [X] 2200550414_63019357
- [X] 2200550471_63020031
- [X] 2200550490_63019351
- [X] 2200550505_63019340
- [X] 2200555565_63018614
- [X] 2200559438_63020029
- [X] 2200559682_63020030
- [X] 2200559713_63019623
- [X] 2200559739_63019626
- [X] 2200569969_63019991
- [X] 2200570001_63021580
- [X] 2200570025_63021490
- [X] 2200570035_63021491
- [X] 2200570042_63021493
- [X] 2200570050_63021494
- [X] 2200579897_63024910
- [X] 2200583995_63024866
- [X] 2200584035_63024905
- [X] 2200584069_63024888
- [X] 2200584126_63024810
- [X] 2200589507_63026712
- [X] 2200597365_63027994
- [X] 2200597480_63027988
- [X] 2200597752_63026853
- [X] 2200597778_63027992
- [X] 22005977_63026903
- [X] 2200609031_63026527
- [X] 2200614198_63113928
- [X] 2200620372_63030821
- [X] 2200620442_63030810
- [X] 2200620498_63030816
- [X] 2200620628_63031031
- [X] 2200622310_63030984
- [X] 2200622355_63030956
- [X] 2200625369_63028699
- [X] 2200625410_63028697
- [X] 2200625536_63028694
- [X] 2200630189_63030665
- [X] 2200635149_63033182
- [X] 2200644544_63037731
- [X] 2200644594_63037725
- [X] 2200650089_63038093
- [X] 2200666292_63076568
- [X] 2200669188_63036688
- [X] 2200669320_63040259
- [X] 2200669383_63040254
- [X] 2200669414_63040257
- [X] 2200669446_63040251
- [X] 2200680342_63105271
- [X] 2200694535_63042853
- [X] 2200694789_63042862
- [X] 2200694858_63042702
- [X] 2200694917_63042696
- [X] 2200699290_63043047
- [X] 2200699345_63040238
- [X] 2200699383_63043050
- [X] 2200699412_63040731
- [X] 220071551_63048935
- [X] 2200731515_63048963
- [X] 2200748145_63051198
- [X] 2200748171_63051213
- [X] 2200751046_63051249
- [X] 2200751101_63051234
- [X] 2200766471_63054590
- [X] 2200767731_63054595
- [X] 2200767822_63054464
- [X] 2200775505_63060410
- [X] 2200850441_63019345
- [X] 220597589_63026879
- [X] 2300003253_63060430
- [X] 2300005679_63060370
- [X] 2300009914_63060390
- [X] 2300028784_63060001
- [X] 2300036815_63063357
- [X] 2300055382_63061874
- [X] 2300055421_63061871
- [X] 2300055440_63061880
- [X] 230006894_63064950
- [X] 2300071111_63070356
- [X] 2300083434_

Replacement in projects/bisonex.org at line 79 [17.35]

B:BD[20.41237] → [20.41237:41706]

∅:D[20.41706] → [23.183:213]

B:BD[14.33083] → [23.183:213]

B:BD[23.213] → [15.35452:39918]

118093 : NA12878
- NA12878 x4
*** DONE Comparer variants cento à sortie bisonex: 50/121 confirmé en sanger, 71/121 non testé, 0 confirmés manqué par pipeline, 5 manqué mais non confirmés
CLOSED: [2023-11-08 Wed 00:19] SCHEDULED: <2023-11-04 Sat>
*** Comparger sanger : variant seul
Compliqué de reconstituer l'arbre familial. L'information est là mais demande du travail.
ON suppose que le variant n'est que dans la famille....
Résultats
❯ open sa
ngerized.csv | where "Found by
 bisonex" == "found" | where "Confirmed in sanger" == "true" | length
50
❯ open sangerized.csv | where "Found by bisonex" == "found" | where "Confirmed in sanger" == "" | length
71
❯ open sangerized.csv | where "Found by bisonex" == "missed" | where "Confirmed in sanger" == "" | length
5
❯ open sangerized.csv | where "Found by bisonex" == "missed" | where "Confirmed in sanger" == "true" | length
0
[[id:cd79a77c-a0b6-4bb1-9e08-fe08dc89e3aa][Résultats finaux]]
*** DONE Regarder 5 variants manqués: 3 explicables, 2 non
CLOSED: [2023-11-09 Thu 00:22] SCHEDULED: <2023-11-05 Sun>
open searched.csv |  where "Found by bisonex" == "missed"
62982193  7884996 : haplotypecaller ok... -> filtré car AD=5 <= 10
63012582  102230760 : non présent haplotypcellar mais une délétion en 755 (en 754 CG -> C). Vérifié mobidetails
63019340  50721335 : non présent haplotypecaller (vérifié igv). vérifié mobidetails
63060439  26869324 : filtré car 15 reads
63109239  14358800 : présent haplotypecaller : filtré car DP=29 <= 30
Non présent haplotypecaller avec bcftools mais zgrep ok
zgrep 7884996 call_variant/haplotypecaller/*62982193*/*
zgrep 102230760 call_variant/haplotypecaller/*63012582*/*
zgrep 50721335 call_variant/haplotypecaller/*63019340*/*
zgrep 26869324 call_variant/haplotypecaller/*63060439*/*
zgrep 14358800 call_variant/haplotypecaller/*63109239*/*
*** DONE Flowchart
CLOSED: [2023-11-09 Thu 00:22]
*** DONE Refaire extraction
CLOSED: [2023-11-04 Sat 19:02] SCHEDULED: <2023-11-04 Sat>
*** DONE Refaire annotation avec mobidetails
CLOSED: [2023-11-04 Sat 19:02] SCHEDULED: <2023-11-04 Sat>
*** DONE Refaire annotation avec transcrit non reconnus
CLOSED: [2023-11-04 Sat 20:42] SCHEDULED: <2023-11-04 Sat>
5 transcrits, donnés égalemen tpar
#+begin_src nu
open annotated.csv | where coding != "negatif" | where chrom == ""
#+end_src
| 62676048 | NM_001080420.1 | SHANK3    | référénce non valide   |
| 62690893 | NM_001080420.1 | KDM6B     | idem                   |
| 62690893 | NM_001080420.1 | KDM6B     | même variant           |
| 62795429 | NM_016381.3    | TREX1 | NM_033629.5   |
| 63019340 | NM_001080420.1 | SHANK3 | NM_001372044.2 |
SCHEDULED: <2023-11-01 Wed>
*** DONE Rajouter variant pour 63009152
CLOSED: [2023-11-04 Sat 20:47] SCHEDULED: <2023-11-01 Wed>
*** DONE Regénérer annotation avec NC_
CLOSED: [2023-11-04 Sat 18:59] SCHEDULED: <2023-10-31 Tue>
*** DONE Comparer variants manqué avec sanger: 0 confirmés
CLOSED: [2023-11-06 Mon 23:48] SCHEDULED: <2023-11-04 Sat>
*** DONE Annoter variants avec sanger
CLOSED: [2023-11-08 Wed 23:17] SCHEDULED: <2023-11-07 Tue>
*** DONE Mail paul avec résultats
CLOSED: [2023-11-09 Thu 00:22] SCHEDULED: <2023-11-05 Sun>
*** DONE Vérifier coordonnées des 2 variants manquants
CLOSED: [2023-11-12 Sun 16:53] SCHEDULED: <2023-11-11 Sat>
Les 2 sont des homopolymer
- 1er = même variant mais représenté différement
- SHANK3 ?
**** PITX3: filtrée car AD=8
NB: représentation synonyme
Même séquence
  >hg38_dna range=chr10:102230742-102230777 5'pad=2 3'pad=2 strand=+ repeatMasking=none
GGAGCCAGCCCGGGGGGGCCCCCGCCCAGGCCCTG
>hg19_dna range=chr10:103990500-103990534 5'pad=0 3'pad=0 strand=+ repeatMasking=none
GGAGCCAGCCCGGGGGGGCCCCCGCCCAGGCCCTG
Selon IGV:
GGAGCCAGCCC(G)GGGGGGCCCCCGCCCAGGCCCTG
Selon cento
GGAGCCAGCCCGGGGGG(G)CCCCCGCCCAGGCCCTG
#+begin_src sh :dir ~/annex/data/bisonex/
bcftools filter -i 'POS=102230760' call_variant/haplotypecaller/*63012582*/*.vcf.gz
#+end_src
DP ok mais AD trop faible
 GT:AD:DP:GQ:PL  0/1:26,8:34:99:146,0,671
**** SHANK3: transcrit supprimé depuis: ok
Retrouvé par ERic: 50721504dup
On vérifie
#+begin_src sh :dir ~/annex/data/bisonex/
bcftools filter -i 'POS=50721504' call_variant/haplotypecaller/*63019340*/*.vcf.gz
#+end_src
#+begin_src sh :dir ~/annex/data/bisonex/
zgrep '50721504' annotate/full/*63019340*.tsv
#+end_src
*** TODO Sanger pour 4 VOUS manqués
SCHEDULED: <2023-12-13 Wed>
/Entered on/ [2023-11-13 Mon 22:40]
* Résultats
** TODO Speed-up BWA-mem
SCHEDULED: <2023-11-26 Sun>
** TODO Speed-up Hapotypecaller
SCHEDULED: <2023-11-26 Sun>
** TODO Refaire statistics avec happy+ vcfeval
SCHEDULED: <2023-11-25 Sat>
/Entered on/ [2023-11-18 Sat 20:13]
* Communication
** DONE Mail NGS-diag
CLOSED: [2023-10-06 Fri 08:04] SCHEDULED: <2023-10-06 Fri>
/Entered on/ [2023-10-04 Wed 19:33]

[20.41237]

118093 : NA12878
- NA12878 x4
*** DONE Comparer variants cento à sortie bisonex: 50/121 confirmé en sanger, 71/121 non testé, 0 confirmés manqué par pipeline, 5 manqué mais non confirmés
CLOSED: [2023-11-08 Wed 00:19] SCHEDULED: <2023-11-04 Sat>
*** Comparger sanger : variant seul
Compliqué de reconstituer l'arbre familial. L'information est là mais demande du travail.
ON suppose que le variant n'est que dans la famille....
Résultats
❯ open sangerized.csv | where "Found by bisonex" == "found" | where "Confirmed in sanger" == "true" | length
50
❯ open sangerized.csv | where "Found by bisonex" == "found" | where "Confirmed in sanger" == "" | length
71
❯ open sangerized.csv | where "Found by bisonex" == "missed" | where "Confirmed in sanger" == "" | length
5
❯ open sangerized.csv | where "Found by bisonex" == "missed" | where "Confirmed in sanger" == "true" | length
0
[[id:cd79a77c-a0b6-4bb1-9e08-fe08dc89e3aa][Résultats finaux]]
*** DONE Regarder 5 variants manqués: 3 explicables, 2 non
CLOSED: [2023-11-09 Thu 00:22] SCHEDULED: <2023-11-05 Sun>
open searched.csv |  where "Found by bisonex" == "missed"
62982193  7884996 : haplotypecaller ok... -> filtré car AD=5 <= 10
63012582  102230760 : non présent haplotypcellar mais une délétion en 755 (en 754 CG -> C). Vérifié mobidetails
63019340  50721335 : non présent haplotypecaller (vérifié igv). vérifié mobidetails
63060439  26869324 : filtré car 15 reads
63109239  14358800 : présent haplotypecaller : filtré car DP=29 <= 30
Non présent haplotypecaller avec bcftools mais zgrep ok
zgrep 7884996 call_variant/haplotypecaller/*62982193*/*
zgrep 102230760 call_variant/haplotypecaller/*63012582*/*
zgrep 50721335 call_variant/haplotypecaller/*63019340*/*
zgrep 26869324 call_variant/haplotypecaller/*63060439*/*
zgrep 14358800 call_variant/haplotypecaller/*63109239*/*
*** DONE Flowchart
CLOSED: [2023-11-09 Thu 00:22]
*** DONE Refaire extraction
CLOSED: [2023-11-04 Sat 19:02] SCHEDULED: <2023-11-04 Sat>
*** DONE Refaire annotation avec mobidetails
CLOSED: [2023-11-04 Sat 19:02] SCHEDULED: <2023-11-04 Sat>
*** DONE Refaire annotation avec transcrit non reconnus
CLOSED: [2023-11-04 Sat 20:42] SCHEDULED: <2023-11-04 Sat>
5 transcrits, donnés égalemen tpar
#+begin_src nu
open annotated.csv | where coding != "negatif" | where chrom == ""
#+end_src
| 62676048 | NM_001080420.1 | SHANK3    | référénce non valide   |
| 62690893 | NM_001080420.1 | KDM6B     | idem                   |
| 62690893 | NM_001080420.1 | KDM6B     | même variant           |
| 62795429 | NM_016381.3    | TREX1 | NM_033629.5   |
| 63019340 | NM_001080420.1 | SHANK3 | NM_001372044.2 |
SCHEDULED: <2023-11-01 Wed>
*** DONE Rajouter variant pour 63009152
CLOSED: [2023-11-04 Sat 20:47] SCHEDULED: <2023-11-01 Wed>
*** DONE Regénérer annotation avec NC_
CLOSED: [2023-11-04 Sat 18:59] SCHEDULED: <2023-10-31 Tue>
*** DONE Comparer variants manqué avec sanger: 0 confirmés
CLOSED: [2023-11-06 Mon 23:48] SCHEDULED: <2023-11-04 Sat>
*** DONE Annoter variants avec sanger
CLOSED: [2023-11-08 Wed 23:17] SCHEDULED: <2023-11-07 Tue>
*** DONE Mail paul avec résultats
CLOSED: [2023-11-09 Thu 00:22] SCHEDULED: <2023-11-05 Sun>
*** DONE Vérifier coordonnées des 2 variants manquants
CLOSED: [2023-11-12 Sun 16:53] SCHEDULED: <2023-11-11 Sat>
Les 2 sont des homopolymer
- 1er = même variant mais représenté différement
- SHANK3 ?
**** PITX3: filtrée car AD=8
NB: représentation synonyme
Même séquence
  >hg38_dna range=chr10:102230742-102230777 5'pad=2 3'pad=2 strand=+ repeatMasking=none
GGAGCCAGCCCGGGGGGGCCCCCGCCCAGGCCCTG
>hg19_dna range=chr10:103990500-103990534 5'pad=0 3'pad=0 strand=+ repeatMasking=none
GGAGCCAGCCCGGGGGGGCCCCCGCCCAGGCCCTG
Selon IGV:
GGAGCCAGCCC(G)GGGGGGCCCCCGCCCAGGCCCTG
Selon cento
GGAGCCAGCCCGGGGGG(G)CCCCCGCCCAGGCCCTG
#+begin_src sh :dir ~/annex/data/bisonex/
bcftools filter -i 'POS=102230760' call_variant/haplotypecaller/*63012582*/*.vcf.gz
#+end_src
DP ok mais AD trop faible
 GT:AD:DP:GQ:PL  0/1:26,8:34:99:146,0,671
**** SHANK3: transcrit supprimé depuis: ok
Retrouvé par ERic: 50721504dup
On vérifie
#+begin_src sh :dir ~/annex/data/bisonex/
bcftools filter -i 'POS=50721504' call_variant/haplotypecaller/*63019340*/*.vcf.gz
#+end_src
#+begin_src sh :dir ~/annex/data/bisonex/
zgrep '50721504' annotate/full/*63019340*.tsv
#+end_src
*** TODO Sanger pour 4 VOUS manqués
SCHEDULED: <2023-12-13 Wed>
/Entered on/ [2023-11-13 Mon 22:40]
* Résultats
** TODO Speed-up BWA-mem
SCHEDULED: <2023-11-26 Sun>
** TODO Speed-up Hapotypecaller
SCHEDULED: <2023-11-26 Sun>
** TODO Refaire statistics avec happy+ vcfeval
SCHEDULED: <2023-11-30 Thu>
/Entered on/ [2023-11-18 Sat 20:13]
* Communication
** DONE Mail NGS-diag
CLOSED: [2023-10-06 Fri 08:04] SCHEDULED: <2023-10-06 Fri>
/Entered on/ [2023-10-04 Wed 19:33]

File move: 20230624220255-youtube_api_requete.org → youtube_api_requete.org
BF:BFD[24.29] → [25.917:979]
BF:BF[25.979] → [25.211:211]
[24.29]
[25.211]
File move: 20230709153124-wiki_rtl8723bu.org → wiki_rtl8723bu.org
BF:BFD[24.29] → [26.10525:10582]
BF:BF[26.10582] → [26.10293:10293]
[24.29]
[26.10293]
File move: 20230806124759-vpn.org → vpn.org
BF:BFD[24.29] → [27.1678:1724]
BF:BF[27.1724] → [27.1495:1495]
[24.29]
[27.1495]
File move: 20230511173148-voyages.org → voyages.org
BF:BFD[24.29] → [28.16196:16246]
BF:BF[28.16246] → [12.116745:116745]
[24.29]
[12.116745]
File move: 20230511180622-voiture.org → voiture.org
BF:BFD[24.29] → [29.5163:5213]
BF:BF[29.5213] → [29.4864:4864]
[24.29]
[29.4864]
File move: 20231118123458-visualisation.org → visualisation.org
BF:BFD[24.29] → [30.1707:1763]
BF:BF[30.1763] → [30.232:232]
[24.29]
[30.232]
File move: 20230511180443-utf_8.org → utf_8.org
BF:BFD[24.29] → [29.5556:5604]
BF:BF[29.5604] → [31.90622:90622]
[24.29]
[31.90622]
File move: 20230511173311-usenet.org → usenet.org
BF:BFD[24.29] → [28.15705:15754]
BF:BF[28.15754] → [28.11768:11768]
[24.29]
[28.11768]
File move: 20230511174932-turbidimetrie.org → turbidimetrie.org
BF:BFD[24.29] → [29.7515:7571]
BF:BF[29.7571] → [32.83:83]
[24.29]
[32.83]
File move: 20230511181024-the_elements_of_statistical_learning.org → the_elements_of_statistical_learning.org
BF:BFD[24.29] → [29.1092:1171]
BF:BF[29.1171] → [33.636:636]
[24.29]
[33.636]
File move: 20230511170628-stockage.org → stockage.org
BF:BFD[24.29] → [28.46049:46100]
BF:BF[28.46100] → [34.1660:1660]
[24.29]
[34.1660]
File move: 20230603093614-sqlite3.org → sqlite3.org
BF:BFD[24.29] → [35.479:529]
BF:BF[35.529] → [35.207:207]
[24.29]
[35.207]
File move: 20230511173213-series.org → series.org
BF:BFD[24.29] → [28.16145:16194]
BF:BF[28.16194] → [28.15756:15756]
[24.29]
[28.15756]

File addition: seedhost.org (----------)

[24.29]

#+title: Seedhost
#+filetags: personal
i
* Rtorrent config
directory.default.set = ~/downloads
schedule2 = watch.directory,5,5,load.start=~/downloads/watch/rtorrent/*.torrent
network.port_range.set = YYYYY
network.port_random.set =no
pieces.hash.on_completion.set = no
pieces.hash.on_completion.set = no
trackers.use_udp.set = yes
encryption = allow_incoming,enable_retry,try_outgoing
dht.mode.set = disable
protocol.pex.set = no
network.http.ssl_verify_peer.set = 0
encoding.add = UTF-8
network.xmlrpc.size_limit.set = 2097152
system.file.max_size.set = -1
session.path.set = ZZZ
network.bind_address.set = XXXX
network.scgi.open_port = XXXX
execute2 = YYYY

File move: 20230511173415-sauvegarde_freebsd.org → sauvegarde_freebsd.org
BF:BFD[24.29] → [28.11426:11487]
BF:BF[28.11487] → [36.283625:283625]
[24.29]
[36.283625]
File move: 20230511181257-recettes.org → recettes.org
BF:BFD[24.29] → [29.933:984]
BF:BF[29.984] → [12.590656:590656]
[24.29]
[12.590656]
File move: 20230514100330-r_nix.org → r_nix.org
BF:BFD[24.29] → [37.32296:32344]
BF:BF[37.32344] → [37.31987:31987]
[24.29]
[37.31987]
File move: 20230514100219-python_nix.org → python_nix.org
BF:BFD[24.29] → [37.34052:34105]
BF:BF[37.34105] → [37.32346:32346]
[24.29]
[37.32346]
File move: 20230721174436-proton_mail.org → proton_mail.org
BF:BFD[24.29] → [38.826:880]
BF:BF[38.880] → [38.215:215]
[24.29]
[38.215]
File move: 20230930172946-projet_evolution.org → projet_evolution.org
BF:BFD[24.29] → [39.18891:18950]
BF:BF[39.18950] → [39.16780:16780]
[24.29]
[39.16780]

File addition: pandoc.org (----------)

[24.29]

* Remplacer liens org-roam pour hakyll
#+begin_src haskell
{-# LANGUAGE TemplateHaskell            #-}
{-# LANGUAGE QuasiQuotes                #-}
{-# LANGUAGE TypeFamilies               #-}
{-# LANGUAGE MultiParamTypeClasses      #-}
{-# LANGUAGE GADTs                      #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE RecordWildCards            #-}
{-# LANGUAGE FlexibleInstances          #-}
{-# LANGUAGE OverloadedStrings          #-}
{-# LANGUAGE DerivingStrategies         #-}
{-# LANGUAGE StandaloneDeriving         #-}
{-# LANGUAGE UndecidableInstances       #-}
{-# LANGUAGE DataKinds #-}
import qualified Database.Persist.TH as PTH
import Database.Persist (Entity(..))
import Database.Persist.Sql (toSqlKey)
import qualified Data.Text as T
import Database.Persist.Sqlite
import Control.Monad.IO.Class
import Control.Monad.Logger
import Text.Pandoc.JSON
import System.FilePath (addExtension, dropExtension, makeRelative)
import System.Directory (getCurrentDirectory)
PTH.share [PTH.mkPersist PTH.sqlSettings, PTH.mkMigrate "migrateAll"] [PTH.persistLowerCase|
  Node sql=nodes
    Id T.Text sql=id
    file T.Text
    title T.Text
    deriving Show
|]
path =  "/home/alex/.emacs.d/.local/cache/org-roam.db"
unescape :: T.Text -> T.Text
unescape = T.replace "\"" ""
-- From "id:XXXX" search in org-roam database for path to file
-- If there is no id, just return the string unchanged
pathFromID :: T.Text -> IO (T.Text)
pathFromID id = runSqlite path $ do
    -- Get id and add (escaped) quote
    let s = T.concat ["\"", last (T.splitOn "id:" id), "\""]
    test <- get (NodeKey s)
    let res = case test of
                Just x -> unescape . nodeFile $ x
                Nothing -> id
    return res
-- Change link to HTML version for publishing it
-- Link is transformed from absolute to relative
-- And we add the root folder for publishing
-- FIXME this will not work locally...
htmlLink :: FilePath -> FilePath -> FilePath
htmlLink f pwd = "/" ++ makeRelative pwd (addExtension (dropExtension f) ".html")
-- Replace org-mode internal link to link to the full path of the file
replaceLink :: Inline -> IO (Inline)
replaceLink (Link attr xs t) = do
  p <- pathFromID (fst t)
  pwd <- getCurrentDirectory
  let p' = htmlLink (T.unpack p) pwd
  return $ Link attr xs (T.pack p', snd t)
replaceLink x = return x
main :: IO ()
main = toJSONFilter replaceLink
#+end_src

File move: 20230514164725-org_roam.org → org_roam.org
BF:BFD[24.29] → [40.332:383]
BF:BF[40.383] → [40.188:188]
[24.29]
[40.188]
File move: 20230511172142-org_mode.org → org_mode.org
BF:BFD[24.29] → [28.17232:17283]
BF:BF[28.17283] → [41.127:127]
[24.29]
[41.127]
File move: ophn1.bib → ophn1.org
BF:BFD[24.29] → [24.708:741]
BF:BF[24.741] → [42.168:168]
[24.29]
[42.168]
Insertion in notes/ophn1.org at line 1 [42.168]
[42.168]
[42.169]
```
#+begin_src bibtex
```
Insertion in notes/ophn1.org at line 41 [42.168]
[42.3256]
```
#+end_src
```
File move: 20230511170524-ocr.org → ocr.org
BF:BFD[24.29] → [28.46203:46249]
BF:BF[28.46249] → [12.745529:745529]
[24.29]
[12.745529]
File move: 20230701162910-nouveau_driver_gentoo.org → nouveau_driver_gentoo.org
BF:BFD[24.29] → [44.2460:2524]
BF:BF[44.2524] → [44.680:680]
[24.29]
[44.680]
File move: 20230511173735-musique.org → musique.org
BF:BFD[24.29] → [28.1372:1422]
BF:BF[28.1422] → [28.1116:1116]
[24.29]
[28.1116]
File move: 20230511180745-microbiologie.org → microbiologie.org
BF:BFD[24.29] → [10.41524:41580]
BF:BF[10.41580] → [29.3565:3565]
[24.29]
[29.3565]
File move: 20230910225228-virologie.org → virologie.org
BF:BFD[47.1207] → [48.4552:4604]
BF:BF[48.4604] → [48.146:146]
[47.1207]
[48.146]
Deletion in notes/medecine/virologie.org at line 1 [48.146]
B:BD[48.146] → [48.147:214]
```
:PROPERTIES:
:ID:       6c2348f1-0081-44d2-974b-1642b20892b7
:END:
```
File move: 20230613000928-tests_bacteriologie.org → tests_bacteriologie.org
BF:BFD[47.1207] → [3.141:203]
BF:BF[3.203] → [49.72:72]
[47.1207]
[49.72]
Deletion in notes/medecine/tests_bacteriologie.org at line 1 [49.72]
B:BD[49.72] → [3.204:271]
```
:PROPERTIES:
:ID:       54934a42-6e6b-4826-9c04-6452588c2e3d
:END:
```
File move: 20230813215947-pneumopathie_a_legionnelle.org → pneumopathie_a_legionnelle.org
BF:BFD[47.1207] → [50.1835:1904]
BF:BF[50.1904] → [50.608:608]
[47.1207]
[50.608]
Deletion in notes/medecine/pneumopathie_a_legionnelle.org at line 1 [50.608]
B:BD[50.608] → [50.609:676]
```
:PROPERTIES:
:ID:       ea96632b-27d0-4c90-9320-d2d648c62ead
:END:
```
File move: 20230625220001-meningites.org → meningites.org
BF:BFD[47.1207] → [51.2727:2780]
BF:BF[51.2780] → [51.142:142]
[47.1207]
[51.142]
Deletion in notes/medecine/meningites.org at line 1 [51.142]
B:BD[51.142] → [51.143:210]
```
:PROPERTIES:
:ID:       7cc7020d-7ac1-42ad-8c90-3b534736924f
:END:
```
File move: 20230528235213-maladies_infectieuses.org → maladies_infectieuses.org
BF:BFD[47.1207] → [52.6421:6485]
BF:BF[52.6485] → [52.3503:3503]
[47.1207]
[52.3503]
Deletion in notes/medecine/maladies_infectieuses.org at line 1 [52.3503]
B:BD[52.3503] → [52.3504:3571]
```
:PROPERTIES:
:ID:       00e9454a-9a71-4fbd-bfde-0fdf323bce15
:END:
```
File move: 20230531000449-infections_urinaires.org → infections_urinaires.org
BF:BFD[47.1207] → [53.3462:3525]
BF:BF[53.3525] → [53.613:613]
[47.1207]
[53.613]
Deletion in notes/medecine/infections_urinaires.org at line 1 [53.613]
B:BD[53.613] → [53.614:681]
```
:PROPERTIES:
:ID:       9347af68-14c8-4bc0-b986-9dc4da51c13d
:END:
```
File move: 20230908231348-infections_cutanees.org → infections_cutanees.org
BF:BFD[47.1207] → [54.2184:2246]
BF:BF[54.2246] → [54.146:146]
[47.1207]
[54.146]
Deletion in notes/medecine/infections_cutanees.org at line 1 [54.146]
B:BD[54.146] → [54.147:214]
```
:PROPERTIES:
:ID:       ae7e200a-30e8-44b5-8566-221968f2e464
:END:
```
File move: 20231105122132-physiologie_hematopoiese.org → hemato.org
BF:BFD[47.1207] → [55.1538:1605]
BF:BF[55.1605] → [55.244:244]
[47.1207]
[55.244]

Replacement in notes/medecine/hemato.org at line 1 [55.244]

B:BD[55.244] → [55.245:349]

:PROPERTIES:
:ID:       0a17eb6d-a88e-4a07-b8af-8435e4086578
:END:
#+title: Physiologie hématopoïèse

[55.244]

[55.349]

#+title: Hématologie

File move: 20230531000409-gastro_enterites.org → gastro_enterites.org
BF:BFD[47.1207] → [53.3527:3586]
BF:BF[53.3586] → [56.206:206]
[47.1207]
[56.206]
Deletion in notes/medecine/gastro_enterites.org at line 1 [56.206]
B:BD[56.206] → [53.3587:3654]
```
:PROPERTIES:
:ID:       a36141db-9bb2-48ff-8c48-f96bbc4aebf6
:END:
```
Deletion in notes/medecine/externat.org at line 195 [5.9360045]
B:BD[5.9372780] → [5.9372780:9372786]
```
:END:
```
File move: 20230803174345-endocardite_infectieuse.org → endocardite_infectieuse.org
BF:BFD[47.1207] → [27.861:927]
BF:BF[27.927] → [27.144:144]
[47.1207]
[27.144]
Deletion in notes/medecine/endocardite_infectieuse.org at line 1 [27.144]
B:BD[27.144] → [27.145:212]
```
:PROPERTIES:
:ID:       c4eaf677-0289-44a3-8223-98888ce157d1
:END:
```
Deletion in notes/medecine/douleur.org at line 5 [5.10098823]
B:BD[5.10098900] → [5.10098900:10098915]
```
  :PROPERTIES:
```
Deletion in notes/medecine/douleur.org at line 6 [5.10098823]
B:BD[5.10098944] → [5.10098944:10098952]
```
  :END:
```
File move: 20230528235124-culture.org → culture.org
BF:BFD[47.1207] → [52.8767:8817]
BF:BF[52.8817] → [52.6487:6487]
[47.1207]
[52.6487]
Deletion in notes/medecine/culture.org at line 1 [52.6487]
B:BD[52.6487] → [52.6488:6555]
```
:PROPERTIES:
:ID:       a8ad4c3b-9f08-4878-8d9d-febddae20069
:END:
```
File move: 20230528225221-classification_bacteries.org → classification_bacteries.org
BF:BFD[47.1207] → [52.24051:24118]
BF:BF[52.24118] → [52.20341:20341]
[47.1207]
[52.20341]
Deletion in notes/medecine/classification_bacteries.org at line 1 [52.20341]
B:BD[52.20341] → [52.20342:20409]
```
:PROPERTIES:
:ID:       6b2bf94d-9539-4a64-b15b-9511aa90772c
:END:
```
File move: 20210430183035-cancers.org → cancers.org
BF:BFD[47.1207] → [47.4350:4400]
BF:BF[47.4400] → [5.10153068:10153068]
[47.1207]
[5.10153068]
Deletion in notes/medecine/bipolaire.org at line 20 [5.10124217]
B:BD[5.10124948] → [5.10124948:10124961]
```
:PROPERTIES:
```
Deletion in notes/medecine/bipolaire.org at line 21 [5.10124217]
B:BD[5.10124976] → [5.10124976:10124982]
```
:END:
```
File move: 20230528225531-bacteries.org → bacteries.org
BF:BFD[47.1207] → [52.20287:20339]
BF:BF[52.20339] → [52.10483:10483]
[47.1207]
[52.10483]
Deletion in notes/medecine/bacteries.org at line 1 [52.10483]
B:BD[52.10483] → [52.10484:10551]
```
:PROPERTIES:
:ID:       9160ba80-117b-4434-acc9-13676a534da0
:END:
```
File move: 20230528235406-antibiotiques.org → antibiotiques.org
BF:BFD[47.1207] → [52.3445:3501]
BF:BF[52.3501] → [52.138:138]
[47.1207]
[52.138]
Deletion in notes/medecine/antibiotiques.org at line 1 [52.138]
B:BD[52.138] → [52.139:206]
```
:PROPERTIES:
:ID:       46dca88b-671f-4f23-a340-5dc564a48659
:END:
```
File move: 20230908230027-angines.org → angines.org
BF:BFD[47.1207] → [54.2797:2847]
BF:BF[54.2847] → [54.2248:2248]
[47.1207]
[54.2248]
Deletion in notes/medecine/angines.org at line 1 [54.2248]
B:BD[54.2248] → [54.2249:2316]
```
:PROPERTIES:
:ID:       aeb5952d-602e-4ec1-8ce2-304825668eb6
:END:
```
Deletion in notes/medecine/20230707211207-ist.org at line 1 [57.99]
B:BD[57.99] → [58.9849:9916]
```
:PROPERTIES:
:ID:       2b2994ab-e0e4-42e3-a3d5-1da638a5a69e
:END:
```
File move: 20230511170842-la_pleiade.org → la_pleiade.org
BF:BFD[24.29] → [28.45283:45336]
BF:BF[28.45336] → [28.17977:17977]
[24.29]
[28.17977]
File move: 20230531095805-julia.org → julia.org
BF:BFD[24.29] → [59.18348:18396]
BF:BF[59.18396] → [59.17964:17964]
[24.29]
[59.17964]
File move: 20230806132415-japonais_configuration.org → japonais_configuration.org
BF:BFD[24.29] → [27.1428:1493]
BF:BF[27.1493] → [27.1000:1000]
[24.29]
[27.1000]
File move: 20230511181321-japonais.org → japonais.org
BF:BFD[24.29] → [29.614:665]
BF:BF[29.665] → [12.353504:353504]
[24.29]
[12.353504]
Deletion in notes/japonais.org at line 5 [12.353504]
B:BD[12.353557] → [12.353557:353588]
B:BD[12.353596] → [12.353596:353597]
```
  :CUSTOM_ID: aya-sensei-intro
```

Replacement in notes/index.org at line 3 [12.368291]

∅:D[55.1643] → [60.148:464]

B:BD[10.41041] → [60.148:464]

- [[id:6b2bf94d-9539-4a64-b15b-9511aa90772c][Classification bactéries]]
- [[id:9160ba80-117b-4434-acc9-13676a534da0][Bactéries]]
- [[id:00e9454a-9a71-4fbd-bfde-0fdf323bce15][Maladies infectieuses]]
- [[id:46dca88b-671f-4f23-a340-5dc564a48659][Antibiotiques]]
- [[id:a8ad4c3b-9f08-4878-8d9d-febddae20069][Culture]]

[55.1643]

[55.1644]

- [[./medecine/bacteries.html][Bactéries]]
- [[./medecine/classification_bacteries.html][Classification bactéries]]
- [[./medecine/maladies_infectieuses.html][Maladies infectieuses]]
- [[./medecine/antibiotiques.html][Antibiotiques]]
- [[./medecine/culture.html][Culture]]

Replacement in notes/index.org at line 9 [12.368291]
∅:D[55.1657] → [48.4633:4690]
B:BD[48.4633] → [48.4633:4690]
```
- [[id:6c2348f1-0081-44d2-974b-1642b20892b7][Virologie]]
```
[55.1657]
[55.1658]
```
- [[./medecine/virologie.html][Virologie]]
```

Replacement in notes/index.org at line 11 [12.368291]

B:BD[55.1674] → [55.1674:1747]

[[id:0a17eb6d-a88e-4a07-b8af-8435e4086578][Physiologie hématopoïèse]]

[55.1674]

[61.223]

- [[./medecine/hematologie.html]]

Replacement in notes/index.org at line 13 [12.368291]
B:BD[61.232] → [6.128285:128341]
```
- [[id:ff3ddbe9-e87b-4e1b-8478-66234ebf6ab5][Japonais]]
```
[61.232]
```
- [[./japonais.html][Japonais]]
```
File move: 20230527220943-haskell_package_pour_gentoo.org → haskell_package_pour_gentoo.org
BF:BFD[24.29] → [52.24181:24251]
BF:BF[52.24251] → [62.1076:1076]
[24.29]
[62.1076]
File move: 20230514100352-haskell_nix.org → haskell_nix.org
BF:BFD[24.29] → [37.31931:31985]
BF:BF[37.31985] → [37.31466:31466]
[24.29]
[37.31466]
File move: 20230522224022-gpg.org → gpg.org
BF:BFD[24.29] → [63.3403:3449]
BF:BF[63.3449] → [63.2755:2755]
[24.29]
[63.2755]
File move: 20230511175421-gnus.org → gnus.org
BF:BFD[24.29] → [29.6886:6933]
BF:BF[29.6933] → [64.4376:4376]
[24.29]
[64.4376]
File move: 20230511175508-git_annex.org → git_annex.org
BF:BFD[24.29] → [29.6697:6749]
BF:BF[29.6749] → [12.374764:374764]
[24.29]
[12.374764]
File move: 20230511170707-gerer_sa_bibliographie.org → gerer_sa_bibliographie.org
BF:BFD[24.29] → [28.45881:45946]
BF:BF[28.45946] → [65.142:142]
[24.29]
[65.142]
File move: 20230511173347-freebsd.org → freebsd.org
BF:BFD[24.29] → [28.11608:11658]
BF:BF[28.11658] → [12.557257:557257]
[24.29]
[12.557257]
File move: 20230511180201-films.org → films.org
BF:BFD[24.29] → [29.6013:6061]
BF:BF[29.6061] → [12.148685:148685]
[24.29]
[12.148685]
File move: 20230511175338-emacs.org → emacs.org
BF:BFD[24.29] → [29.7106:7154]
BF:BF[29.7154] → [12.577073:577073]
[24.29]
[12.577073]
File move: 20230511175552-divers.org → divers.org
BF:BFD[24.29] → [29.6315:6364]
BF:BF[29.6364] → [12.152435:152435]
[24.29]
[12.152435]
File move: 20230511171905-corde_a_sauter.org → corde_a_sauter.org
BF:BFD[24.29] → [28.17520:17577]
BF:BF[28.17577] → [12.353201:353201]
[24.29]
[12.353201]
File move: 20230624144548-buku.org → buku.org
BF:BFD[24.29] → [67.690:737]
BF:BF[67.737] → [67.211:211]
[24.29]
[67.211]
File move: 20230930161424-backup.org → backup.org
BF:BFD[24.29] → [69.49675:49724]
BF:BF[69.49724] → [69.46554:46554]
[24.29]
[69.46554]
File move: 20230511173825-bach.org → bach.org
BF:BFD[24.29] → [28.1067:1114]
BF:BF[28.1114] → [28.377:377]
[24.29]
[28.377]
File move: 20230702145728-alacritty.org → alacritty.org
BF:BFD[24.29] → [44.626:678]
BF:BF[44.678] → [44.215:215]
[24.29]
[44.215]
File move: 20230511173900-a_partager.org → a_partager.org
BF:BFD[24.29] → [28.215:268]
BF:BF[28.268] → [12.139882:139882]
[24.29]
[12.139882]
File move: 20230511170807-a_lire.org → a_lire.org
BF:BFD[24.29] → [28.45830:45879]
BF:BF[28.45879] → [28.45338:45338]
[24.29]
[28.45338]
File move: 20230511173710-a_ecouter.org → a_ecouter.org
BF:BFD[24.29] → [28.11225:11277]
BF:BF[28.11277] → [28.1424:1424]
[24.29]
[28.1424]

File addition: inbox.org_archive (----------)

[5.15272215]

#    -*- mode: org -*-
Archived entries from file /home/alex/roam/personal/inbox.org
* DONE Scraper Lonely Planet
CLOSED: [2023-11-19 Sun 00:01] SCHEDULED: <2023-11-18 Sat>
:PROPERTIES:
:ARCHIVE_TIME: 2023-11-20 Mon 21:39
:ARCHIVE_FILE: ~/roam/personal/inbox.org
:ARCHIVE_CATEGORY: inbox
:ARCHIVE_TODO: DONE
:ARCHIVE_ITAGS: inbox
:END:
:LOGBOOK:
CLOCK: [2023-11-18 Sat 21:58]
:END:
/Entered on/ [2023-11-18 Sat 21:57]

Replacement in Shakefile.hs at line 7 [70.844]
B:BD[70.1017] → [6.128968:129003]
B:BD[6.129003] → [71.1821:1856]
```
filterExe = "_build/filterOrgRoam"
args = "-s --css /css/default.css"
```
[70.1017]
[6.129003]
```
-- args = "-s --css /css/default.css"
```
Deletion in Shakefile.hs at line 18 [70.844]
∅:D[60.3110] → [72.452:452]
∅:D[6.129139] → [72.452:452]
B:BD[73.1730] → [72.452:452]
∅:D[72.452] → [70.1383:1384]
∅:D[74.1638] → [70.1383:1384]
∅:D[73.1730] → [70.1383:1384]
∅:D[75.6857] → [70.1383:1384]
B:BD[70.1383] → [70.1383:1384]
B:BD[70.1384] → [6.129140:129183]
∅:D[6.129183] → [60.3155:3191]
B:BD[60.3155] → [60.3155:3191]
B:BD[60.3191] → [6.129184:129240]
B:BD[6.129240] → [76.55089:55176]
∅:D[76.55176] → [6.129303:129396]
B:BD[6.129303] → [6.129303:129396]
B:BD[6.129396] → [77.7337:7400]
```
    "_site/notes/index.html" %> \out -> do
        let src = "notes/index.org"
        -- Only org roam notes (starting with the date)
        org <- getDirectoryFiles "" ["notes/medecine/20*.org", "notes/*japonais*.org"]
        let html = ["_site" </> n -<.> "html" | n <- org]
        need $ html ++ [filterExe]
        cmd "pandoc" src "--filter " filterExe args "-o" [out]
```
Deletion in Shakefile.hs at line 19 [70.844]
B:BD[70.1496] → [2.469:509]
∅:D[2.509] → [60.3440:3490]
∅:D[76.55216] → [60.3440:3490]
∅:D[6.129530] → [60.3440:3490]
B:BD[60.3440] → [60.3440:3490]
B:BD[60.3490] → [78.33774:33793]
B:BD[78.33793] → [77.7401:7466]
∅:D[77.7466] → [60.3593:3594]
∅:D[6.129617] → [60.3593:3594]
B:BD[60.3593] → [60.3593:3594]
```
    "_site/notes//*.html" %> \out -> do
        let org = dropDirectory1 $ out -<.> "org"
        need [org]
        cmd "pandoc" [org] "--filter " filterExe args "-o" [out]
```

Replacement in Shakefile.hs at line 20 [70.844]

B:BD[6.129645] → [6.129645:129694]

      cmd_ "ghc --make -o" [out] ["src/Main.hs"]

[6.129645]

[60.3623]

      let hs = "src/Main.hs"
      need [hs]
      cmd_ "ghc --make -o" [out] hs

Deletion in Shakefile.hs at line 24 [70.844]
B:BD[60.3624] → [6.129695:129782]
∅:D[6.129782] → [60.3654:3655]
B:BD[60.3654] → [60.3654:3655]
```
    filterExe  %> \out -> do
      cmd_ "ghc --make -o" [out] ["src/filterOrgRoam.hs"]
```
Replacement in Shakefile.hs at line 26 [70.844]
B:BD[6.129842] → [78.33794:33829]
```
        need [siteExe, filterExe ]
```
[6.129842]
[6.129865]
```
        need [siteExe]
```

Replacement in Shakefile.hs at line 44 [70.844]

B:BD[6.130154] → [78.33881:33957]

        need ["build", "_site/notes/index.html", "_site/notes/livres.html"]

[6.130154]

[6.130203]

        need ["build"]

Generating all notes with hakyll (dumping org-roam)

Dependencies

In channels

Change contents

File deletion: site.tar.gz

Insertion in src/Main.hs at line 10 [8.9907]

Insertion in src/Main.hs at line 33 [8.9907]

Insertion in src/Main.hs at line 45 [8.9907]

Replacement in projects.org at line 77 [12.123895]

Replacement in projects.org at line 82 [12.123895]

Insertion in projects.org at line 88 [12.123895]

Replacement in projects.org at line 104 [12.123895]

Insertion in projects.org at line 109 [12.123895]

Replacement in projects.org at line 398 [12.123895]

Replacement in projects.org at line 422 [12.123895]

Insertion in projects.org at line 700 [12.123895]

Replacement in projects/bisonex.org at line 30 [17.35]

Replacement in projects/bisonex.org at line 63 [17.35]

Replacement in projects/bisonex.org at line 77 [17.35]

Replacement in projects/bisonex.org at line 79 [17.35]

File move: 20230624220255-youtube_api_requete.org → youtube_api_requete.org

File move: 20230709153124-wiki_rtl8723bu.org → wiki_rtl8723bu.org

File move: 20230806124759-vpn.org → vpn.org

File move: 20230511173148-voyages.org → voyages.org

File move: 20230511180622-voiture.org → voiture.org

File move: 20231118123458-visualisation.org → visualisation.org

File move: 20230511180443-utf_8.org → utf_8.org

File move: 20230511173311-usenet.org → usenet.org

File move: 20230511174932-turbidimetrie.org → turbidimetrie.org

File move: 20230511181024-the_elements_of_statistical_learning.org → the_elements_of_statistical_learning.org

File move: 20230511170628-stockage.org → stockage.org

File move: 20230603093614-sqlite3.org → sqlite3.org

File move: 20230511173213-series.org → series.org

File addition: seedhost.org (----------)

File move: 20230511173415-sauvegarde_freebsd.org → sauvegarde_freebsd.org

File move: 20230511181257-recettes.org → recettes.org

File move: 20230514100330-r_nix.org → r_nix.org

File move: 20230514100219-python_nix.org → python_nix.org

File move: 20230721174436-proton_mail.org → proton_mail.org

File move: 20230930172946-projet_evolution.org → projet_evolution.org

File addition: pandoc.org (----------)

File move: 20230514164725-org_roam.org → org_roam.org

File move: 20230511172142-org_mode.org → org_mode.org

File move: ophn1.bib → ophn1.org

Insertion in notes/ophn1.org at line 1 [42.168]

Insertion in notes/ophn1.org at line 41 [42.168]

File move: 20230511170524-ocr.org → ocr.org

File move: 20230701162910-nouveau_driver_gentoo.org → nouveau_driver_gentoo.org

File move: 20230511173735-musique.org → musique.org

File move: 20230511180745-microbiologie.org → microbiologie.org

File move: 20230910225228-virologie.org → virologie.org

Deletion in notes/medecine/virologie.org at line 1 [48.146]

File move: 20230613000928-tests_bacteriologie.org → tests_bacteriologie.org

Deletion in notes/medecine/tests_bacteriologie.org at line 1 [49.72]

File move: 20230813215947-pneumopathie_a_legionnelle.org → pneumopathie_a_legionnelle.org

Deletion in notes/medecine/pneumopathie_a_legionnelle.org at line 1 [50.608]

File move: 20230625220001-meningites.org → meningites.org

Deletion in notes/medecine/meningites.org at line 1 [51.142]

File move: 20230528235213-maladies_infectieuses.org → maladies_infectieuses.org

Deletion in notes/medecine/maladies_infectieuses.org at line 1 [52.3503]

File move: 20230531000449-infections_urinaires.org → infections_urinaires.org

Deletion in notes/medecine/infections_urinaires.org at line 1 [53.613]

File move: 20230908231348-infections_cutanees.org → infections_cutanees.org

Deletion in notes/medecine/infections_cutanees.org at line 1 [54.146]

File move: 20231105122132-physiologie_hematopoiese.org → hemato.org

Replacement in notes/medecine/hemato.org at line 1 [55.244]

File move: 20230531000409-gastro_enterites.org → gastro_enterites.org

Deletion in notes/medecine/gastro_enterites.org at line 1 [56.206]

Deletion in notes/medecine/externat.org at line 195 [5.9360045]

File move: 20230803174345-endocardite_infectieuse.org → endocardite_infectieuse.org

Deletion in notes/medecine/endocardite_infectieuse.org at line 1 [27.144]

Deletion in notes/medecine/douleur.org at line 5 [5.10098823]

Deletion in notes/medecine/douleur.org at line 6 [5.10098823]

File move: 20230528235124-culture.org → culture.org

Deletion in notes/medecine/culture.org at line 1 [52.6487]

File move: 20230528225221-classification_bacteries.org → classification_bacteries.org

Deletion in notes/medecine/classification_bacteries.org at line 1 [52.20341]

File move: 20210430183035-cancers.org → cancers.org

Deletion in notes/medecine/bipolaire.org at line 20 [5.10124217]

Deletion in notes/medecine/bipolaire.org at line 21 [5.10124217]