Write a comment

Peer Review of "An expanded method for malaria parasite genetic surveillance using targeted nanopore sequencing"

I. PAPER SUMMARY

Harrott and colleagues describe an improved molecular surveillance method for malaria parasites using nanopore sequencing. Malaria remains a significant global health burden with 250 million cases and over 600,000 deaths, predominantly affecting children in sub-Saharan Africa. The authors develop "DRAG2," an expanded version of their previous multiplex PCR assay, which now includes additional gene targets for antimalarial drug resistance markers, full-length merozoite surface protein 2 (msp2), and 18S ribosomal RNA for Plasmodium species identification. Testing on 122 clinical samples from Ghana demonstrated that DRAG2 provides more even coverage across targets compared to the original assay. The assay successfully detected drug resistance markers and distinguished between Plasmodium species in both artificial plasmid mixtures and clinical samples. However, several methodological limitations requiring additional clarification, such as assay optimization, quality control thresholds, and comparative performance metrics, need to be addressed. If these recommendations are implemented, it will greatly improve the manuscript and the impact of the work. 

II. MAJOR REVISIONS

Multiplex PCR Design and Optimization 

• Experimental Evidence for Assay Optimization

Critique: The paper would benefit from providing more comprehensive experimental evidence supporting the optimization process for the DRAG2 multiplex PCR assay. While the authors mention "in vitro testing of primer combinations to identify high-performing combinations based on PCR product inspection by agarose gel electrophoresis and nanopore sequencing," they do not present the systematic optimization data that would be expected for a robust molecular diagnostic assay development process.

Framework for addressing: 

• Provide a supplementary table or figure showing the systematic optimization process, including primer concentration matrices tested for both DRAG2-A and DRAG2-B reactions.

• Present quantitative metrics (band intensity quantification or amplicon coverage statistics) used to determine the optimal reaction.

• Include experimental data justifying the decision to split the assay into two separate multiplex reactions.

• The rationale for Dividing into Two Multiplex Reactions

Critique: The authors' decision to divide the DRAG2 assay into two separate multiplex reactions (DRAG2-A and DRAG2-B) requires stronger experimental justification. They state this approach "reduced the risk of primer interactions and increased target specificity," but do not provide comparative data demonstrating performance issues in a single multiplex reaction that would necessitate this division (Elnifro et al., 2000). The addition of a second PCR reaction doubles the complexity of the workflow. 

Relevant section: "We divided the assay into two separate multiplex reactions, which we refer to as DRAG2-A and DRAG2-B. This reduced the risk of primer interactions and increased target specificity."

Framework for addressing: 

•  Present side-by-side comparison data showing failed or suboptimal amplification in a single-tube format versus the two-tube format. 

•  Provide quantitative metrics of improved specificity (e.g., reduction in non-specific bands, improved coverage uniformity) resulting from splitting the reactions. 

• Conduct a cost-benefit analysis comparing the increased complexity/cost against the performance improvements. 

  Justification for Coverage Thresholds

Critique: The coverage thresholds established for quality control in the DRAG2 assay require more robust statistical justification. The authors state they "established a pragmatic cut-off for clinical samples to be at least 7.5x the coverage of negative controls" and set "an absolute threshold of 50x coverage per amplicon," but do not provide analytical validation for these specific thresholds (Jennings et al., 2017). Particularly concerning is the 50x coverage threshold for nanopore sequencing, which has higher error rates compared to other platforms and may require substantially higher coverage for reliable variant calling (Wick et al., 2019). Without a systematic analysis of variant calling accuracy at different coverage depths specific to this assay, these thresholds appear to be selected without evidence-based validation.

Relevant section: "We established a pragmatic cut-off for clinical samples to be at least 7.5x the coverage of negative controls for each amplicon in the run to pass QC filters, on the basis that this would 'pass' all the positive controls tested across multiple sequencing runs. We also used an absolute coverage threshold of 50x per amplicon, and if a sample has three or more amplicons that fail QC (out of the 10 amplicons in the assay) then the whole sample failed."

Framework for addressing: 

• Perform systematic analysis of variant calling accuracy at various coverage depths (e.g., 25x, 50x, 100x, 200x) using samples with known variants to establish an evidence-based minimum coverage threshold.

• Conduct a ROC curve analysis to determine the optimal fold-change threshold relative to negative controls that maximize sensitivity and specificity.

• Evaluate the impact of coverage on minor allele detection in mixed infections by testing artificial mixtures at different ratios with various coverage depths.

• Assess whether different amplicons require different minimum coverage thresholds based on sequence complexity or error profiles. 

• Justify the "three failed amplicons" rule through analysis of the correlation between amplicon failures and overall sample quality.

  Statistical Comparison with DRAG1

Critique: The paper would be strengthened by including formal statistical analyses to support claims of improved performance of DRAG2 compared to DRAG1. The authors state that "amplicon coverage for the DRAG2 assay was more even than for DRAG1" and that drug resistance marker frequencies "were consistent with the frequencies calculated from the DRAG1 assay," but do not provide statistical significance testing for these comparisons (Giavarina, 2015). Without quantitative metrics and appropriate statistical tests, it is difficult to objectively evaluate the magnitude and significance of the claimed improvements.

Relevant section: "Amplicon coverage for the DRAG2 assay was more even than for DRAG1. Median coverage across all DRAG2 amplicon targets, including those that failed quality control (QC) filtering, was 10,727x (interquartile range (IQR), 3,745-24,774); including only QC-pass amplicons, median coverage was 11,781x (IQR, 4938-26,925)." and "Drug resistance marker frequencies were calculated from the variants called from the amplicon sequence data (Table 5). These were consistent with the frequencies calculated from the DRAG1 assay reported in Ref. 5."

Framework for addressing: 

• Conduct paired statistical analyses (e.g., paired t-tests or Wilcoxon signed-rank tests) on the overlapping set of 95 samples tested with both assays to compare coverage metrics.

• Calculate and report a standardized measure of coverage uniformity (e.g., coefficient of variation across amplicons) for both assays. 

• Provide scatter plots or Bland-Altman plots comparing key performance metrics between DRAG1 and DRAG2 to visualize the nature and magnitude of differences

III. MINOR REVISIONS

Analysis of msp2 for Multiplicity of Infection

Relevant section: "Full-length msp2 was included due to its polymorphism, with potential to act as an approximate indicator of a multiplicity of infection (MOI) i.e. infections with multiple P. falciparum clones." 

Critique: The inclusion of the msp2 gene as a marker for a multiplicity of infection (MOI) requires a more detailed explanation of the analytical framework. The authors state that msp2 has the "potential to act as an approximate indicator of a multiplicity of infection," but do not elaborate on how they analyze msp2 polymorphisms for this purpose.  

Framework for addressing: Describe the specific bioinformatic approach for analyzing msp2 sequence data to identify distinct allelic variants. 

• Define clear criteria for classifying infections as monoclonal or polyclonal based on msp2 sequence data. 

• Specify thresholds for minor variant detection (e.g., minimum read count or percentage)

  Workflow Visualization

Critique: The manuscript would benefit from a comprehensive workflow figure illustrating the complete DRAG2 assay process from sample to data analysis. Workflow diagrams are particularly valuable for complex molecular assays with multiple steps, helping readers conceptualize the entire process and understand the relationship between different components (Schubert et al., 2016). Given the complexity of the DRAG2 assay, involving multiple PCR reactions, library preparation, nanopore sequencing, and bioinformatic analysis, a visual representation would significantly enhance the clarity and accessibility of the methodology. 

Framework for addressing: 

• Create a flowchart diagram showing all major steps from sample collection through final data analysis.

• Include decision points and quality control checkpoints within the workflow

• Clearly distinguish between the DRAG2-A and DRAG2-B pathways and where they converge. 

• Consider a multi-panel figure showing: (A) an Overall workflow schematic, (B) Amplicon target locations on gene maps, (C) an Example of a quality control decision tree, (D) a Bioinformatic pipeline overview.

Recommendation: Major and Minor Revisions Required.

Competing interests

The authors declare that they have no competing interests.

Use of Artificial Intelligence (AI)

The authors declare that they used generative AI to come up with new ideas for their review.