# Analyze with online GREAT

#### Zuguang Gu ( z.gu@dkfz.de )

#### 2022-11-21

Note: On Aug 19 2019 GREAT released version 4 which supports `hg38` genome and removes some ontologies such pathways. `submitGreatJob()` still takes `hg19` as default. `hg38` can be specified by argument `species = "hg38"`. To use the older versions such as 3.0.0, specify as `submitGreatJob(..., version = "3")`.

GREAT (Genomic Regions Enrichment of Annotations Tool) is a popular web-based tool to associate biological functions to genomic regions. The rGREAT package makes GREAT anlaysis automatic by first constructing a HTTP POST request according to user’s input and retrieving results from GREAT web server afterwards.

## Submit the job

Load the package:

    library(rGREAT)

The input data is either a `GRanges` object or a BED-format data frame, no matter it is sorted or not. In following example, we use a `GRanges` object which is randomly generated.

    set.seed(123)
    gr = randomRegions(nr = 1000, genome = "hg19")
    head(gr)

    ## GRanges object with 6 ranges and 0 metadata columns:
    ##       seqnames            ranges strand
    ##          <Rle>         <IRanges>  <Rle>
    ##   [1]     chr1   9204434-9208784      *
    ##   [2]     chr1   9853594-9859363      *
    ##   [3]     chr1 10862809-10871681      *
    ##   [4]     chr1 12716970-12723206      *
    ##   [5]     chr1 13814692-13823250      *
    ##   [6]     chr1 19243285-19247097      *
    ##   -------
    ##   seqinfo: 26 sequences from an unspecified genome; no seqlengths

Submit genomic regions by `submitGreatJob()`.

The returned variable `job` is a `GreatJob` class instance which can be used to retrieve results from GREAT server and store results which are already downloaded.

    job = submitGreatJob(gr)

You can get the summary of your job by directly printing `job`.

    job

    ## Submit time: 2022-06-05 16:10:27
    ##   Note the results may only be avaiable on GREAT server for 24 hours.
    ## Version: 4.0.4
    ## Species: hg19
    ## Inputs: 3748 regions
    ## Mode: Basal plus extension
    ##   Proximal: 5 kb upstream, 1 kb downstream,
    ##   plus Distal: up to 1000 kb
    ## Include curated regulatory domains
    ##
    ## Enrichment tables for following ontologies have been downloaded:
    ##   GO Biological Process
    ##   GO Cellular Component
    ##   GO Molecular Function

More parameters can be set for the job:

    job = submitGreatJob(gr, species = "mm9") # of course, gr should be from mm9
    job = submitGreatJob(gr, adv_upstream = 10, adv_downstream = 2, adv_span = 2000)
    job = submitGreatJob(gr, rule = "twoClosest", adv_twoDistance = 2000)
    job = submitGreatJob(gr, rule = "oneClosest", adv_oneDistance = 2000)

Also you can choose different versions of GREAT for the analysis.

    job = submitGreatJob(gr, version = "3.0")
    job = submitGreatJob(gr, version = "2.0")

Note: from rGREAT package 1.99.0, background by `bg` argument is not supported any more (currently you can still use it, but you will see a warning message), because GREAT requires a special format for `gr` and `bg` if both are set, and it uses a different method for the enrichment analysis and returns enrichment tables in a different format. But still, you can use local GREAT to integrate background regions. Seel the rGREAT paper for more details.

Available parameters are (following content is copied from GREAT website):

- `species`: “hg38”, “hg19”, “mm10”, “mm9” are supported in GREAT version 4.x.x, “hg19”, “mm10”, “mm9”, “danRer7” are supported in GREAT version 3.x.x and “hg19”, “hg18”, “mm9”, “danRer7” are supported in GREAT version 2.x.x.
- `includeCuratedRegDoms`: Whether to include curated regulatory domains.
- `rule`: How to associate genomic regions to genes.
  - `basalPlusExt`: mode ‘Basal plus extension’. Gene regulatory domain definition: Each gene is assigned a basal regulatory domain of a minimum distance upstream and downstream of the TSS (regardless of other nearby genes). The gene regulatory domain is extended in both directions to the nearest gene’s basal domain but no more than the maximum extension in one direction.
    - `adv_upstream`: proximal extension to upstream (unit: kb)
    - `adv_downstream`: proximal extension to downstream (unit: kb)
    - `adv_span`: maximum extension (unit: kb)
  - `twoClosest`: mode ‘Two nearest genes’. Gene regulatory domain definition: Each gene is assigned a regulatory domain that extends in both directions to the nearest gene’s TSS but no more than the maximum extension in one direction.
    - `adv_twoDistance`: maximum extension (unit: kb)
  - `oneClosest`: mode ‘Single nearest gene’. Gene regulatory domain definition: Each gene is assigned a regulatory domain that extends in both directions to the midpoint between the gene’s TSS and the nearest gene’s TSS but no more than the maximum extension in one direction.
    - `adv_oneDistance`: maximum extension (unit: kb)

GREAT uses the UCSC bed-format where genomic coordinates are 0-based. Many R packages generate genomic regions as 1-based. Thus by default, the start positions of regions are subtracted by 1. If your regions are already 0-based, you can specify `gr_is_zero_based = TRUE` in `submitGreatJob()`. Anyway in most cases, this will only slightly affect the enrichment results.

## Get enrichment tables

With `job`, we can now retrieve results from GREAT. The first and the primary results are the tables which contain enrichment statistics for the analysis. By default it will retrieve results from three GO Ontologies. All tables contains statistics for all terms no matter they are significant or not. Users can then make filtering with self-defined cutoff.

There is a column for adjusted p-values by “BH” method. Other p-value adjustment methods can be applied by `p.adjust()`.

The returned value of `getEnrichmentTables()` is a list of data frames in which each one corresponds to the table for a single ontology. The structure of data frames are same as the tables on GREAT website.

    tbl = getEnrichmentTables(job)

    ## The default enrichment tables contain no associated genes for the input
    ## regions.You can set `download_by = 'tsv'` to download the complete
    ## table,but note only the top 500 regions can be retreived. See the
    ## following link:
    ##
    ## https://great-help.atlassian.net/wiki/spaces/GREAT/pages/655401/Export#Export-GlobalExport
    ##
    ## or you can try the local GREAT analysis with the function `great()`.

    names(tbl)

    ## [1] "GO Molecular Function" "GO Biological Process" "GO Cellular Component"

    tbl[[1]][1:2, ]

    ##           ID            name Binom_Genome_Fraction Binom_Expected
    ## 1 GO:0005515 protein binding             0.7515051       2816.641
    ## 2 GO:0005488         binding             0.8694407       3258.664
    ##   Binom_Observed_Region_Hits Binom_Fold_Enrichment Binom_Region_Set_Coverage
    ## 1                       3186              1.131135                 0.8500534
    ## 2                       3505              1.075594                 0.9351654
    ##   Binom_Raw_PValue Binom_Adjp_BH Hyper_Total_Genes Hyper_Expected
    ## 1     2.435386e-49  1.027489e-45             11129       2302.717
    ## 2     4.308462e-39  9.088701e-36             14268       2952.212
    ##   Hyper_Observed_Gene_Hits Hyper_Fold_Enrichment Hyper_Gene_Set_Coverage
    ## 1                     2635              1.144300               0.6865555
    ## 2                     3159              1.070045               0.8230849
    ##   Hyper_Term_Gene_Coverage Hyper_Raw_PValue Hyper_Adjp_BH
    ## 1                0.2367688     1.078774e-35  4.551348e-32
    ## 2                0.2214045     5.578685e-20  1.176824e-16

Information stored in `job` will be updated after retrieving enrichment tables.

    job

    ## Submit time: 2022-06-05 16:10:27
    ##   Note the results may only be avaiable on GREAT server for 24 hours.
    ## Version: 4.0.4
    ## Species: hg19
    ## Inputs: 3748 regions
    ## Mode: Basal plus extension
    ##   Proximal: 5 kb upstream, 1 kb downstream,
    ##   plus Distal: up to 1000 kb
    ## Include curated regulatory domains
    ##
    ## Enrichment tables for following ontologies have been downloaded:
    ##   GO Biological Process
    ##   GO Cellular Component
    ##   GO Molecular Function

You can get results by either specifying the ontologies or by the pre-defined categories (categories already contains pre-defined sets of ontologies):

    tbl = getEnrichmentTables(job, ontology = c("GO Molecular Function", "Human Phenotype"))
    tbl = getEnrichmentTables(job, category = c("GO"))

As you have seen in the previous messages and results, The enrichment tables contain no associated genes. However, you can set `download_by = 'tsv'` in `getEnrichmentTables()` to download the complete tables, but due to the restriction from GREAT web server, only the top 500 regions can be retreived (check the last two columns of `tbl2[["GO Molecular Function"]]` in the following example).

    tbl2 = getEnrichmentTables(job, download_by = "tsv")

All available ontology names for a given species can be get by `availableOntologies()` and all available ontology categories can be get by `availableCategories()`. Here you do not need to provide species information because `job` already contains it.

    availableOntologies(job)

    ## [1] "GO Molecular Function"     "GO Biological Process"
    ## [3] "GO Cellular Component"     "Mouse Phenotype"
    ## [5] "Mouse Phenotype Single KO" "Human Phenotype"
    ## [7] "Ensembl Genes"

    availableCategories(job)

    ## [1] "GO"        "Phenotype" "Genes"

    availableOntologies(job, category = "GO")

    ## [1] "GO Molecular Function" "GO Biological Process" "GO Cellular Component"

## Make volcano plot

In differential gene expression analysis, volcano plot is used to visualize relations between log2 fold change and (adjusted) p-values. Similarly, we can also use volcano plot to visualize relations between fold enrichment and (adjusted) p-values for the enrichment analysis. The plot is made by the function `plotVolcano()`:

    plotVolcano(job, ontology = "GO Biological Process")

As the enrichment analysis basically only looks for over-representations, it is actually half volcano.

## Get region-gene associations

Association between genomic regions and genes can be plotted by `plotRegionGeneAssociations()`. The function will make the three plots which are same as on GREAT website.

    plotRegionGeneAssociations(job)

`getRegionGeneAssociations()` returns a `GRanges` object which contains the gene-region associations. Note the column `dist_to_TSS` is based on the middle points of the input regions to TSS.

    getRegionGeneAssociations(job)

    ## GRanges object with 3734 ranges and 2 metadata columns:
    ##          seqnames              ranges strand |        annotated_genes
    ##             <Rle>           <IRanges>  <Rle> |        <CharacterList>
    ##      [1]     chr1     1310672-1311002      * |               AURKAIP1
    ##      [2]     chr1     1342595-1342925      * |                 MRPL20
    ##      [3]     chr1     1509773-1510103      * |                  SSU72
    ##      [4]     chr1     1711637-1711967      * |                   NADK
    ##      [5]     chr1     2323001-2323331      * |                   RER1
    ##      ...      ...                 ...    ... .                    ...
    ##   [3730]     chrX 130912591-130912921      * | ENSG00000134602,OR13H1
    ##   [3731]     chrX 130947194-130947524      * | ENSG00000134602,OR13H1
    ##   [3732]     chrX 134449748-134450078      * |            CT55,ZNF75D
    ##   [3733]     chrX 138877458-138877788      * |            MCF2,ATP11C
    ##   [3734]     chrX 147691650-147691980      * |               AFF2,IDS
    ##             dist_to_TSS
    ##           <IntegerList>
    ##      [1]           -300
    ##      [2]            -67
    ##      [3]            311
    ##      [4]          -1893
    ##      [5]           -106
    ##      ...            ...
    ##   [3730] -244537,234806
    ##   [3731] -209934,269409
    ##   [3732]  -144591,28099
    ##   [3733]   -87237,36824
    ##   [3734]  109676,895062
    ##   -------
    ##   seqinfo: 23 sequences from an unspecified genome; no seqlengths

Please note the two meta columns are in formats of `CharacterList` and `IntegerList` because a region may associate to multiple genes.

You can also choose only plotting one of the three figures.

    plotRegionGeneAssociations(job, which_plot = 1)

By specifying ontology and term ID, you can get the associations in a certain term. Here the term ID is from the first column of the data frame from `getEnrichmentTables()`.

    plotRegionGeneAssociations(job, ontology = "GO Molecular Function",
        term_id = "GO:0004984")

    getRegionGeneAssociations(job, ontology = "GO Molecular Function",
        term_id = "GO:0004984")

    ## GRanges object with 12 ranges and 2 metadata columns:
    ##        seqnames              ranges strand | annotated_genes   dist_to_TSS
    ##           <Rle>           <IRanges>  <Rle> | <CharacterList> <IntegerList>
    ##    [1]     chr1 248851932-248852262      * |          OR14I1         -6468
    ##    [2]     chr5 180618737-180619067      * |           OR2V2         36959
    ##    [3]     chr5 180626146-180626476      * |           OR2V2         44368
    ##    [4]    chr11   57559484-57559814      * |           OR9Q1       -231704
    ##    [5]    chr14   20763236-20763566      * |          OR11H4         52503
    ##    ...      ...                 ...    ... .             ...           ...
    ##    [8]    chr14   22961072-22961402      * |           OR4E2        827940
    ##    [9]    chr14   23005816-23006146      * |           OR4E2        872684
    ##   [10]    chr14   23025349-23025580      * |           OR4E2        892168
    ##   [11]     chrX 130912591-130912921      * |          OR13H1        234806
    ##   [12]     chrX 130947194-130947524      * |          OR13H1        269409
    ##   -------
    ##   seqinfo: 5 sequences from an unspecified genome; no seqlengths

## The Shiny application

`shinyReport()` creates a Shiny application to view the complete results:

    shinyReport(job)

## Session info

    sessionInfo()

    ## R version 4.2.2 (2022-10-31 ucrt)
    ## Platform: x86_64-w64-mingw32/x64 (64-bit)
    ## Running under: Windows Server x64 (build 20348)
    ##
    ## Matrix products: default
    ##
    ## locale:
    ## [1] LC_COLLATE=C
    ## [2] LC_CTYPE=English_United States.utf8
    ## [3] LC_MONETARY=English_United States.utf8
    ## [4] LC_NUMERIC=C
    ## [5] LC_TIME=English_United States.utf8
    ##
    ## attached base packages:
    ## [1] stats4    stats     graphics  grDevices utils     datasets  methods
    ## [8] base
    ##
    ## other attached packages:
    ##  [1] TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2
    ##  [2] GenomicFeatures_1.50.2
    ##  [3] AnnotationDbi_1.60.0
    ##  [4] Biobase_2.58.0
    ##  [5] rGREAT_2.0.2
    ##  [6] GenomicRanges_1.50.1
    ##  [7] GenomeInfoDb_1.34.3
    ##  [8] IRanges_2.32.0
    ##  [9] S4Vectors_0.36.0
    ## [10] BiocGenerics_0.44.0
    ## [11] knitr_1.41
    ##
    ## loaded via a namespace (and not attached):
    ##   [1] bitops_1.0-7
    ##   [2] matrixStats_0.63.0
    ##   [3] bit64_4.0.5
    ##   [4] doParallel_1.0.17
    ##   [5] filelock_1.0.2
    ##   [6] RColorBrewer_1.1-3
    ##   [7] progress_1.2.2
    ##   [8] httr_1.4.4
    ##   [9] rprojroot_2.0.3
    ##  [10] tools_4.2.2
    ##  [11] bslib_0.4.1
    ##  [12] utf8_1.2.2
    ##  [13] R6_2.5.1
    ##  [14] DT_0.26
    ##  [15] DBI_1.1.3
    ##  [16] colorspace_2.0-3
    ##  [17] GetoptLong_1.0.5
    ##  [18] withr_2.5.0
    ##  [19] processx_3.8.0
    ##  [20] tidyselect_1.2.0
    ##  [21] prettyunits_1.1.1
    ##  [22] bit_4.0.5
    ##  [23] curl_4.3.3
    ##  [24] compiler_4.2.2
    ##  [25] cli_3.4.1
    ##  [26] xml2_1.3.3
    ##  [27] DelayedArray_0.24.0
    ##  [28] rtracklayer_1.58.0
    ##  [29] sass_0.4.3
    ##  [30] callr_3.7.3
    ##  [31] rappdirs_0.3.3
    ##  [32] stringr_1.4.1
    ##  [33] digest_0.6.30
    ##  [34] Rsamtools_2.14.0
    ##  [35] rmarkdown_2.18
    ##  [36] XVector_0.38.0
    ##  [37] pkgconfig_2.0.3
    ##  [38] htmltools_0.5.3
    ##  [39] MatrixGenerics_1.10.0
    ##  [40] highr_0.9
    ##  [41] dbplyr_2.2.1
    ##  [42] fastmap_1.1.0
    ##  [43] htmlwidgets_1.5.4
    ##  [44] rlang_1.0.6
    ##  [45] GlobalOptions_0.1.2
    ##  [46] RSQLite_2.2.18
    ##  [47] shiny_1.7.3
    ##  [48] shape_1.4.6
    ##  [49] jquerylib_0.1.4
    ##  [50] BiocIO_1.8.0
    ##  [51] generics_0.1.3
    ##  [52] jsonlite_1.8.3
    ##  [53] BiocParallel_1.32.1
    ##  [54] BioMartGOGeneSets_0.99.8
    ##  [55] dplyr_1.0.10
    ##  [56] RCurl_1.98-1.9
    ##  [57] magrittr_2.0.3
    ##  [58] GO.db_3.16.0
    ##  [59] GenomeInfoDbData_1.2.9
    ##  [60] Matrix_1.5-3
    ##  [61] Rcpp_1.0.9
    ##  [62] fansi_1.0.3
    ##  [63] lifecycle_1.0.3
    ##  [64] stringi_1.7.8
    ##  [65] yaml_2.3.6
    ##  [66] SummarizedExperiment_1.28.0
    ##  [67] zlibbioc_1.44.0
    ##  [68] pkgbuild_1.3.1
    ##  [69] org.Hs.eg.db_3.16.0
    ##  [70] BiocFileCache_2.6.0
    ##  [71] grid_4.2.2
    ##  [72] blob_1.2.3
    ##  [73] promises_1.2.0.1
    ##  [74] parallel_4.2.2
    ##  [75] crayon_1.5.2
    ##  [76] lattice_0.20-45
    ##  [77] Biostrings_2.66.0
    ##  [78] circlize_0.4.15
    ##  [79] hms_1.1.2
    ##  [80] KEGGREST_1.38.0
    ##  [81] ps_1.7.2
    ##  [82] pillar_1.8.1
    ##  [83] rjson_0.2.21
    ##  [84] codetools_0.2-18
    ##  [85] biomaRt_2.54.0
    ##  [86] XML_3.99-0.12
    ##  [87] glue_1.6.2
    ##  [88] evaluate_0.18
    ##  [89] remotes_2.4.2
    ##  [90] BiocManager_1.30.19
    ##  [91] httpuv_1.6.6
    ##  [92] foreach_1.5.2
    ##  [93] png_0.1-7
    ##  [94] vctrs_0.5.1
    ##  [95] assertthat_0.2.1
    ##  [96] cachem_1.0.6
    ##  [97] xfun_0.35
    ##  [98] mime_0.12
    ##  [99] xtable_1.8-4
    ## [100] restfulr_0.0.15
    ## [101] later_1.3.0
    ## [102] tibble_3.1.8
    ## [103] TxDb.Hsapiens.UCSC.hg38.knownGene_3.16.0
    ## [104] iterators_1.0.14
    ## [105] GenomicAlignments_1.34.0
    ## [106] memoise_2.0.1
    ## [107] ellipsis_0.3.2