Mckay/c2pro reports test #99

mbowcut2 · 2024-08-16T21:13:38Z

changes for CRISPRessoReports/c2pro-reports

* D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com>

commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <mbowcut@gmail.com> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <mbowcut@gmail.com> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <Cole@colelyman.com> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <Cole@colelyman.com> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <Cole@colelyman.com> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <Cole@colelyman.com> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <Cole@colelyman.com> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <Cole@colelyman.com> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <github@daihan.me> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-…

commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <mbowcut@gmail.com> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <mbowcut@gmail.com> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <Cole@colelyman.com> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <Cole@colelyman.com> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <Cole@colelyman.com> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <Cole@colelyman.com> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <Cole@colelyman.com> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <Cole@colelyman.com> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <github@daihan.me> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] …

Colelyman

This looks great, I can't see anything that needs to be changed! Thanks for doing this.

commit bb7533405af9f355ee9b367903e1f391375c7ca8 Merge: 354f962 2eff6a1 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Aug 22 13:47:21 2024 -0600 Merge branch 'master' into c2pro-reports commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <mbowcut@gmail.com> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <mbowcut@gmail.com> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <Cole@colelyman.com> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <Cole@colelyman.com> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <Cole@colelyman.com> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <Cole@colelyman.com> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <Cole@colelyman.com> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <Cole@colelyman.com> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <github@daihan.me> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no t…

commit ae22c92e10c757667847a043eddef5a2fe333be7 Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 22 16:12:08 2024 -0600 fix cup download commit f4f8a9027a4742b883c562e31fc7fe5f45a5f8cf Merge: 2eff6a1 bb75334 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Aug 22 15:50:20 2024 -0600 Merge pull request #24 from edilytics/c2pro-reports C2pro reports commit bb7533405af9f355ee9b367903e1f391375c7ca8 Merge: 354f962 2eff6a1 Author: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Date: Thu Aug 22 13:47:21 2024 -0600 Merge branch 'master' into c2pro-reports commit 354f962d4201ae4784108b0a89467796af6ec326 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:49:32 2024 -0600 specify encoding commit 823250b8108ede27b29d59d1239c74c93d74395f Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:46:17 2024 -0600 changed _ROOT to _root commit fb8cfa7614598f23e578b55b92ca9b685eef3156 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:44:32 2024 -0600 added docstrings commit e0777a805e4da9729118b429fb44687ecc15d590 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:32:34 2024 -0600 fixes for linting commit 3ade759b85992484ae59aa7db3f10fe8e6bbdc50 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 16:04:43 2024 -0600 Squashed commit of the following: commit 42be4bb0bcd58cdf9c902a448a7e5bde888fb1d3 Author: mbowcut2 <mbowcut@gmail.com> Date: Mon Aug 19 14:36:55 2024 -0600 added pro check for plotly import in batchReport commit 31873d15fff9b9b12c1e8d8c62efa686c12cbd37 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 15:05:58 2024 -0600 pointed integration tests at test branch commit d449600561074c8721322e94a8375b3ad742a88c Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 16 14:24:03 2024 -0600 Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <mbowcut@gmail.com> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <mbowcut@gmail.com> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <Cole@colelyman.com> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <Cole@colelyman.com> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <Cole@colelyman.com> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <Cole@colelyman.com> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <Cole@colelyman.com> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <Cole@colelyman.com> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <github@daihan.me> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension…

* Fix CRISPRessoAggregate bug and other improvements (#95) * D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- * Add amplicon_name to plot functions * Add sgRNA sequences to nucleotide quilt parameters in Aggregate * Add custom_colors to Aggregate plot functions * Update Aggregate and make_aggregate_report to have logger and tool * Write command_used to Aggregate .json info file * Point to new test branch and add Aggregate run * Make the order of Aggregate runs explicit * Sort all instances of crispresso2_folder_info in Aggregate * Sort df_summary_quantification df in Aggregate * Try sorting with a list of single column * Update to correct test branch * Move to master test branch --------- * Squashed commit of the following: commit 6ec98a05ee70f85b5aa0ac15ab6094b7f1f20d08 Author: mbowcut2 <mbowcut@gmail.com> Date: Tue Aug 13 16:44:39 2024 -0600 dict key changes commit 7cfd5acf06da4eb6f49453144ee1fed1e1488a7a Author: mbowcut2 <mbowcut@gmail.com> Date: Thu Aug 8 15:30:31 2024 -0600 added C2PRO install check back commit bfb0003329ea61b5c79c7e1df8d9a73ec5a508db Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 13:08:12 2024 -0600 fixed key error conditionals commit 84444e7480605206cb3efa4a0db675c55e717304 Author: mbowcut2 <mbowcut@gmail.com> Date: Fri Aug 2 09:22:44 2024 -0600 use local jinja_paritals file commit 71dd12786fec6c4aba0170a3bfd9022b06f5eede Author: mbowcut2 <mbowcut@gmail.com> Date: Wed Jul 31 14:10:29 2024 -0600 Squashed commit of the following: commit 5e3b30515c4bc437127e7fb21f53cb0bd511c4ca Author: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Date: Mon Jul 22 09:31:44 2024 -0600 D3-Enhancements (#78) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Fix typo and move flexiguide to debug (#77) * Change flexiguide output to debug level * Fix typo in fastp merged output file name * Adding id tags for d3 script enhancements * pointing to test branch * Add amplicon_name parameter to allele heatmap and line plots * Add function to extract quantification window regions from include_idxs * Scale the quantification window according to the coordinates of the sgRNA plot * added c2pro check, added space in args.json * Correct the quantification window indexes for multiple guides * Fix name of nucleotide conversion plot when guides are not the same * Fix jinja variables that aren't found * Fix multiple guide errors where the wrong sgRNA sequence was associated in d3 plot * Remove unneeded variable and extra whitespace * Switch test branch to master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> commit 09e5d9720ad21e44fc7916d71bde3fd7a9dfa7ef Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu Jul 18 14:31:54 2024 -0600 Asymmetrical cut point (#457) * add cut_point_ind to plot_alleles_heatmap for asymmetrical plotting * Cole asymmetrical cut point (#453) * Pin versions of numpy and matplotlib in CI environment (#84) (#452) * Reduce duplication and implement cut_point_ind in plot_alleles_heatmap_hist --------- Co-authored-by: Cole Lyman <Cole@colelyman.com> commit 8d92972694ddff629dad844a6ad100459f69751d Author: Cole Lyman <Cole@colelyman.com> Date: Thu Jul 18 14:29:40 2024 -0600 Cole/update args (#85) (#456) commit 44f692ecabf5e2eb96ee0cfd7bae62343da7810c Author: Cole Lyman <Cole@colelyman.com> Date: Mon Jul 15 16:17:29 2024 -0600 Implement new pooled mixed-mode default behavior (#454) * changes for pooled mixed-mode default (#83) * changes for pooled mixed-mode default * deprecated old arg * added integration tests for mixed mode * fixed test target * updated test name * pinned numpy * Fix integration tests yml * pinning matplotlib * added print to CI tests * changed mixed mode info string * Remove pooled-mixed-mode-align-to-genome step from Github Actions * Update demultiplex_genome_wide parameter and help * Convert args.json to unix line endings * Add Pooled mixed mode demux run * Update the name of the argument in Pooled * Point integration tests back to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Revert change to pooled mixed mode info statement (#86) --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 79b482b55a0e8edbc03ec22bd2714bade1e90323 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Jul 9 12:53:23 2024 -0600 Pin versions of numpy and matplotlib in CI environment (#84) (#452) commit 80dc1bdd72d50f989717bfc5f8156bc3495c45f4 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 14:07:42 2024 -0600 Add padding to image commit 381755daf0939aaf2745df0a802c809633aff47d Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 30 13:59:57 2024 -0600 White background for schematic for dark mode commit d649db71e610bd8840fbb8d46fadb07789b67390 Author: Cole Lyman <Cole@colelyman.com> Date: Fri May 24 12:45:53 2024 -0600 Fix typo and move flexiguide to debug (#77) (#438) * Change flexiguide output to debug level * Fix typo in fastp merged output file name commit 71181f50ef2b39015523b1a71d9fd1bf0dce14eb Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 13:34:00 2024 -0600 Prefix the release Docker tag with a `v` (#434) commit d2c2be18a6bb64b0e742cc24c4665980a24324bc Author: Cole Lyman <Cole@colelyman.com> Date: Mon May 13 09:41:32 2024 -0600 Showing sgRNA sequences on hover in CRISPRessoPro (#432) * Passing sgRNA sequences to regular and Batch D3 plots (#73) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Provide sgRNA_sequences to plot_nucleotide_quilt plots * Passing sgRNA_sequences to plot * Refactor check for determining when to use CRISPREssoPro or matplotlib for Batch plots * Add max-height to Batch report samples * Change testing branch * Fix wrong check for large Batch plots * Update integration_tests.yml to point back at master --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Push new releases to ECR (#74) * Create aws_ecr.yml (#1) * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * us-east-1 * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Update aws_ecr.yml * Fix d3 sgRNA sequences (#76) * Pass correct sgRNA_sequences to d3 plot * Pass correct sgRNA sequence to prime editor plot for d3 * Resize plotly (#75) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file * Pass div id for plotly * Remove debug --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1c504274818b6b17fb60620d48fd92cb2e50566d Author: Cole Lyman <Cole@colelyman.com> Date: Thu May 9 14:16:25 2024 -0600 Fix plots and improve plot error handling (#431) * Sam/try plots (#71) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Point test to try-plots * Fix d3 not showing and plotly mixing with matplotlib * Use logger for warnings and debug statements * Point tests back at master --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Sam/fix plots (#72) * Fix batch mode pandas warning. (#70) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Functional * Cole/fix status file name (#69) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> * Try catch all futures * Fix test fail plots * Fix d3 not showing and plotly mixing with matplotlib --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Remove token from integration tests file --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit acb2ea8e26dff4cd11f71301b344f81b1cec9040 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Thu May 2 13:49:33 2024 -0600 Use recent docker image for CircleCI testing that includes updated pandas commit 38fd76dbd7ce2087468f9f454b548777de959a68 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 16:42:28 2024 -0600 Cole/fix status file name (#69) (#430) * Update config file logging messages This removes printing the exception (which is essentially a duplicate), and adds a condition if no config file was provided. Also changes `json` to `config` so that it is more clear. * Fix divide by zero when no amplicons are present in Batch mode * Don't append file_prefix to status file name * Place status files in output directories * Update tests branch for file_prefix addition * Load D3 and plotly figures with pro with multiple amplicons * Update batch * Fix bug in CRISPRessoCompare with pointing to report datas with file_prefix Before this fix, when using a file_prefix the second run that was compared would not be displayed as a data in the first figure of the report. * Import CRISPRessoPro instead of importing the version When installed via conda, the version is not available * Remove `get_amplicon_output` unused function from CRISPRessoCompare Also remove unused argparse import * Implement `get_matching_allele_files` in CRISPRessoCompare and accompanying unit tests * Allow for matching of multiple guides in the same amplicon * Fix pandas FutureWarning * Change test branch back to master --------- Co-authored-by: Sam <snic9004@gmail.com> commit 3ec22e5fd09e432c9997d30e5f9ee51a2cc00d7b Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed May 1 13:08:11 2024 -0600 Remove linked space in readme commit 340a4e16795a5e500411e11572ec267525985009 Author: Cole Lyman <Cole@colelyman.com> Date: Wed May 1 13:07:14 2024 -0600 Fix batch mode pandas warning. (#70) (#429) * refactor to call method on DataFrame, rather than Series. Removes warning. * Fix pandas future warning in CRISPRessoWGS --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> commit 1bc9e906f0ded81f80761d1ec375ee50a4f882a9 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 26 16:26:27 2024 -0600 Bump version to 2.3.1 and change default CRISPRessoPooled behavior to change in 2.3.2 (#428) commit 5638a1f6ffa973231f23422e9c757fa8cd4af7cc Author: Kendell Clement <k.clement.dev@gmail.com> Date: Wed Apr 24 18:00:43 2024 -0600 Spelling fixes commit d6011f29db16d8fc1c1e7222457b7f9a1f671de6 Author: Cole Lyman <Cole@colelyman.com> Date: Wed Apr 24 09:33:53 2024 -0600 Extract `jinja_partials` and fix CRISPRessoPooled fastp errors (#425) * Updated README (#64) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Extract jinja_partials (#65) * Extract jinja_partials code * Remove Plotly dependency from setup.py * Fix CRISPRessoPooled flash errors (#68) * Fix replacing flash intermediate files with fastp intermediate files This also moves where the files are added to `files_to_remove` up to near where they are created. * Update to run test branch with paired end Pooled test * Add pooled-paired-sim test to integration tests * Replace flash and trimmomatic with fastp and remove plotly from Github Actions environment * Change test branch back to master --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit f4858a30c43374f54058b3ad9c1e965e1ab7fb46 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 23 17:00:28 2024 -0600 Updated README (#64) (#424) * Updating README to fix argument, email, and formatting * removing superfluous files * Add link to CRISPRessoPro, move CRISPRessoPro section to end, and fix JSON formatting * Remove link to CRISPRessoPro * Replace Docker badge with link to tags * Add bullet points to Guardrails section and improve formatting * Fix typo and removed colons from guardrails --------- Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> commit c3dbff0fccd44b0b1a9c246dd2aa629ddc515787 Author: Kendell Clement <k.clement.dev@gmail.com> Date: Mon Apr 22 11:24:59 2024 -0600 Update CRISPRessoPooledCORE.py (#423) Fix bug in error reporting if duplicate names are present commit 20903c14877e5166b1b8a7b50b8fcab450ea3ca6 Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 16:55:39 2024 -0600 Remove extra imports from CRISPRessoCore (#67) (#422) commit 4aae57e5be475cd717792265bee36a71a99425de Author: Cole Lyman <Cole@colelyman.com> Date: Thu Apr 18 10:00:19 2024 -0600 Cole/refactor jinja undefined (#66) (#421) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Refactor logging Jinja2 undefined variable warnings * Revert plot_11a update * Update intedration test branch * Update jinja to warn on undefined but not fail. Fix all undefined warnings * Fix github integration tests ref * One more undefined variable --------- Co-authored-by: Samuel Nichols <Snic9004@gmail.com> commit 768c3c05bf1786a2a32e135b6e145cd6503c3db1 Author: Cole Lyman <Cole@colelyman.com> Date: Tue Apr 9 17:30:10 2024 -0600 Fix Jinja2 undefined variables (#63) (#417) * Replace Jinja2 PackageLoader with FileSystemLoader The PackageLoader doesn't work with a fairly recent version of Jinja2 (3.0.1) and Python 3.9. Replacing with FileSystemLoader work with the older version and the latest version. * Fix undefined variable `amplicon_name` in report template * Revert plot_11a update * Update intedration test branch * Update branch for integration tests commit 7e18f08cc1ac5f247a0fd1bbb394ccd9b0a07c2e Author: Han Dai <github@daihan.me> Date: Fri Apr 5 18:36:41 2024 -0400 fix: change all U+00A0 to U+0020 (#400) commit 235dc29c0cd0fcca2e999148d4660acf00b07221 Author: Cole Lyman <Cole@colelyman.com> Date: Fri Apr 5 16:36:16 2024 -0600 Fastp, args as data, guardrails, and PE fix (#415) * Change CRISPResso_status.txt format to JSON (#46) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * add json read for status file * changed Formatter to json format * fixed json access variable name: message * changed perentage_complete to numeric * changed status file to .json * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * New makefile commands * changed file to .json * changed status to json file * Make JSON human readable by adding new lines * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * point to test branch * pointed CI config to testing branch * Update integration_tests.yml point to master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Trevor/fastp integration (#50) * Update check_program to check versions and create check_fastq function * Update fastq arg, implement fastp in get_most_frequent_reads * Bump version to 2.3.0 * Deprecate Flash and Trimmomatic parameters, and update fastp params * Update guess_amplicons and guess_guides to remove max_paired_end_reads_overlap * Implement trimming of single end reads * Merge (and trim) reads in CRISPRessoCORE with fastp * Modify error handling to account for fastp errors * Replace flash and trimmomatic with fastp in Docker dependencies * Update LICENSE.txt with fastp info * Remove min and max amplicon length (no longer needed) * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Implement trimming with fastp in CRISPRessoPooled * Implemend merging (and trimming) with fastp in CRISPRessoPooled * Fixed minor fastp errors * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Update where the test point to * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * 3.4->2.08 * Put ttf-mscorefonts-installer back above apt-get clean * restore slash, replace fastp with trimmomatic and flash, add autoremove step --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * initial readme modifications * Updated readme to remove deprecated commands, updated help text to reflect new version and fastp * Pointing test branch back at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com> * Guardrails clean history (#34) * Include guardrail functions * Add CRISPRessoReports subtree * Refactor to use CRISPRessoReports module * Include guardrail functions * Functional guardrails, needs reports update * Add guardrail partial * fix guardrials partial * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman <cole@colelyman.com> * Update C cythonized files * Add exact numbers to guardrails printouts * Remove extraneous whitespace from CRISPRessoCOREResources.pyx * Fix calculation of `total_mods` from being negative The issue was that `all_deletion_coordinates` just tells you how many deletions were present, but not how long the deletion is. * Changes to message * Remove old tag * Point tests at guardrails * Restore C2 pro check * Save message with guardrail name * Point tests repo at master --------- Co-authored-by: Cole Lyman <cole@colelyman.com> Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> * Fix case sensitivity in Prime Editing mode (#54) * Move read filtering to after merging in CRISPResso (#39) * Move read filtering to after merging This is in an effort to be consistent with the behavior and results of CRISPRessoPooled. * Properly assign the correct file names for read filtering * Add space around operators * GitHub actions on pr (#51) * Run integration tests on pull_request * Run pytest on pull_request * Run pylint on pull_request * Run tests on PR only when opening PR (#53) * Update reports (#52) * Update report changes * Switch branch of integration test repo * Remove extraneous `crispresso_data_path` * Point integration tests back to master * Make all amplicons in amplicon_seq_arr uppercase This fixes https://github.com/pinellolab/CRISPResso2/issues/396 * Allow RNA values to be provided for prime_editing_pegRNA_scaffold_seq * Fix 'Prime-edited' key not found (#32) * Move 'Prime-edited' amplicon name check By moving this, it will check if there is an amplicon named 'Prime-edited' (which is a reserved name) even if the `prime_editing_pegRNA_extension_seq` parameter is empty. * Only search for scaffold integration when pegRNA extension seq is provided * Remove spaces at the end of lines * Docker size (#49) * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman <cole@colelyman.com> --------- Co-authored-by: Cole Lyman <cole@colelyman.com> * GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns ----… Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Trevor Martin <60452953+trevormartinj7@users.noreply.github.com> Co-authored-by: Samuel Nichols <Snic9004@gmail.com>

Colelyman and others added 6 commits August 9, 2024 15:27

pointed integration tests at test branch

31873d1

added pro check for plotly import in batchReport

42be4bb

Merge branch 'master' into mckay/c2pro-reports-test

48a3858

Colelyman self-requested a review August 22, 2024 19:04

Colelyman approved these changes Aug 22, 2024

View reviewed changes

mbowcut2 added 3 commits August 22, 2024 14:07

point to tests master

404d6dc

mbowcut2 merged commit b8b9085 into master Aug 22, 2024
3 checks passed

mbowcut2 deleted the mckay/c2pro-reports-test branch August 22, 2024 22:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mckay/c2pro reports test #99

Mckay/c2pro reports test #99

mbowcut2 commented Aug 16, 2024

Colelyman left a comment

Mckay/c2pro reports test #99

Mckay/c2pro reports test #99

Conversation

mbowcut2 commented Aug 16, 2024

Colelyman left a comment

Choose a reason for hiding this comment