(19 intermediate revisions by the same user not shown)
Line 1:
Line 1:
<div style="color:red">
LoadBatch provides the ability to conveinently load a set of genomes or experiments in a single operation. To load a set of genomes or experiments using [[LoadGenome]] and [[LoadExperiment]] would require running the tool for each genome/experiment individually and is very time consuming for large data sets. [[File:LoadBatch.png|thumb|400px]]
UNDER CONSTRUCTION
</div>
LoadBatch provides the ability to conveinently load a set of genomes or experiments in a single operation. To load a set of genomes using [[LoadGenome]] would require running the tool for each genome individually.
== Inputs ==
[[File:LoadBatch.png|thumb|400px]]
=== Metadata File ===
== Inputs ==
A single metadata file that describes the data files contained is required for the load. See the metadata section: '''[[Metadata]]'''
=== Data File(s)===
=== Metadata ===
Data files can be given individually or together as a compressed tar archive file (ending in .tar.gz, also known as a "tarball").
'''Valid combinations of input files include:'''
* tarball of metadata file and data file(s)
* metadata file and tarball of data file(s)
* separate metadata file and data files
=== Data File ===
<span style="color:red">''Note: tarballs must not contain subdirectories.''</span>
You can select and retrieve data file located at:
'''The interface allows you to select and retrieve data files located at:'''
*The iPlant Data Store
*The iPlant Data Store
*An FTP server
*An FTP server
*Your computer (Upload)<br>
*Your computer (Upload)
=== Data Formats and Track Types ===
LoadExperiment supports several data file formats depending on the data type:
*Quantitative data [[File:quant_track.png|thumb|200px|Quantitative track]]
**Comma-separated (CSV) file format
**Tab-separated (TSV) file format
**BED file format
*Marker data [[File:marker_track.png|thumb|200px|Marker track]]
** GFF/GTF file format
*Polymorphism (SNP) data [[File:snp_track.png|thumb|200px|SNP track]]
**Variant Call Format (VCF) file format
*Alignment data [[File:alignment_track.png|thumb|200px|Alignment track]]
**BAM file format
Each of these file formats are described below in their own section. The file type can be auto-detected by LoadExperiment if the file name ends with the expected extension (.csv, .tsv, .bed, .gff, .gtf, .vcf, .bam). Files can be compressed (.zip, .gz) and still have their type auto-detected (e.g., mydata.bed.gz). For non-standard file name extensions, you can select the file type from a list.
==== CSV File Format ====
This is a comma-delimited file that contains the following columns
*Chromosome (string)
*Start position (integer)
*Stop position (integer)
*Chromosome Strand (1 or -1)
*Measurement Value must be between [1-0] (real number; inclusive)
*Second Value (OPTIONAL): can store a second value such as an expect value (real number)
Same as CSV format but with tab delimiters instead of commas.
==== BED File Format ====
Standard BED format as defined here: http://genome.ucsc.edu/FAQ/FAQformat.html#format1
Only the first six columns are used, with the "name" field ignored.
==== GFF File Format ====
Standard GFF3 format as defined here: http://gmod.org/wiki/GFF3
Only the seqid, start, end, score, strand, and attribute columns are used (column numbers 1, 4, 5, 6, 7, 9 respectively).
==== VCF File Format ====
Standard VCF 4.1 format as defined here: http://www.1000genomes.org/wiki/Analysis/Variant%20Call%20Format/vcf-variant-call-format-version-41
==== BAM File Format ====
Standard BAM format.
====FASTQ Data====
=== Data Formats ===
[[EPIC-CoGe]] now supports fastq data generated by RNASeq. When loaded, EPIC-CoGe will run and the [[Expression Analysis Pipeline]] developed by James Schnable for his [http://qteller.com qTeller] project.
For supported '''genome''' data file formats, see '''[[LoadGenome]]'''.
==Bulk Loading==
For supported '''experiment''' data file formats, see '''[[LoadExperiment]]'''.
Please contact the [mailto:coge.genome@gmail.com CoGe Team] if you have large numbers of experiments you wish to load and we can help you with the bulk loading.
Latest revision as of 18:58, 30 March 2015
LoadBatch provides the ability to conveinently load a set of genomes or experiments in a single operation. To load a set of genomes or experiments using LoadGenome and LoadExperiment would require running the tool for each genome/experiment individually and is very time consuming for large data sets.
