---
title: "Data Projection"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Data Projection}
  %\VignetteEncoding{UTF-8}
  %\VignetteEngine{knitr::rmarkdown}
editor_options: 
  markdown: 
    wrap: 72
---

```{r setup, warning=FALSE, message=FALSE}
library(cyCONDOR)
```

Considering the high number of samples that can be generated with modern HDC instruments and the number of cells acquired we developed a data projection workflow in `cyCONDOR`. With this approach an initial model it trained including a UMAP dimensionality reduction and a cell classifier of the cell labels. We can then project any new sample into the pre-trained model, this operation is much faster and allows to analyse millions of cell in few minutes.

# Loading the data for training

We start by loading the data for the training.

```{r}
condor_train <- prep_fcd(data_path = "../.test_files/DataProjection/fcs_train/", 
                         max_cell = 5000, 
                         useCSV = FALSE, 
                         transformation = "auto_logi", 
                         remove_param = c("FSC-H", "SSC-H", "FSC-W", "SSC-W", "Time", "live_dead"), 
                         anno_table = "../.test_files/DataProjection/metadata_train.csv", 
                         filename_col = "filename")
```

```{r}
condor_train$anno$cell_anno$group <- "train"
```

# Loading the data for projection

We also load the data to later project.

```{r}
condor_test <- prep_fcd(data_path = "../.test_files/DataProjection/fcs_test/",
                        max_cell = 10000,
                        useCSV = FALSE,
                        transformation = "auto_logi",
                        remove_param = c("FSC-H", "SSC-H", "FSC-W", "SSC-W", "Time", "live_dead"),
                        anno_table = "../.test_files/DataProjection/metadata_test.csv",
                        filename_col = "filename")
```

```{r}
condor_test$anno$cell_anno$group <- "test"
```

# UMAP Projection

We start now by running a UMAP, in this case we set the `ret_model` variable to `TRUE` to keep the UMAP model in the `condor`  object. The UMAP calculation and data projection can be performed only based on the protein expression (`expr`) as `pca` would be performed independently in the two dataset not providing consistent results.

## Run UMAP keeping the model

```{r}
condor_train <- runUMAP(fcd = condor_train, 
                        input_type = "expr", 
                        data_slot = "orig", 
                        nThreads = 4,
                        ret_model = TRUE)
```

## Add data to the embedding

We can now predict the UMAP coordinates of the test data.

```{r}
condor_test<- learnUMAP(fcd = condor_test,
                        input_type = "expr", 
                        data_slot = "orig", 
                        fcd_model = condor_train,
                        nEpochs = 100, 
                        nThreads = 4, 
                        prefix = "pred")
```

The predicted UMAP coordinates can be accessed via `condor_test$umap$pred_expr_orig`.
```{r}
condor_test$umap$pred_expr_orig[1:5,]
```

# Train a classifier for the label transfer

To transfer also the labels from the reference to the projected data we need to train a cell classifier. We start by clustering the training data. In this case both `FlowSOM` and `Phenogpraph` can be used as input for the cell label kNN classifier. In this vignette we use `Phenograph`.

```{r}
condor_train <- runPhenograph(fcd = condor_train, 
                              input_type = "expr", 
                              data_slot = "orig", 
                              k = 150)
```

We can visualize the `Phenograph` clustering in a UMAP.

```{r}
plot_dim_red(fcd= condor_train,  
             expr_slot = NULL,
             reduction_method = "umap", 
             reduction_slot = "expr_orig", 
             cluster_slot = "phenograph_expr_orig_k_150",
             param = "Phenograph",
             title = "Phenograph clustering of the training data set")
```

# Label transfer

Now, we train the classifier on the clustering labels. If you assigned a metacluster label, this can also be used to train the classifier.

## Train label transfer kNN classifier

Here, we use the `Phenograph` clustering labels as an example to train the classifier. In many cases you probably want to use the metacluster labels of an annotated flow cytometry data set which had been previously assigned using `metaclustering()`.

```{r}
condor_train <- train_transfer_model(fcd = condor_train, 
                                     data_slot = "orig", 
                                     input_type = "expr", 
                                     cluster_slot = "phenograph_expr_orig_k_150",
                                     cluster_var = "Phenograph",
                                     method = "knn", 
                                     tuneLength = 5, 
                                     trControl = caret::trainControl(method = "cv"))
```

```{r}
condor_train$extras$lt_model$performance_plot
```

```{r, fig.height=10, fig.width=10}
#kNN importance
condor_train$extras$lt_model$features_plot
```


## Predict the labels

Based on the trained classifier, we predict now the cluster labels for the test data set.
```{r}
condor_test <- predict_labels(fcd = condor_test, 
                              data_slot = "orig", 
                              input_type = "expr", 
                              fcd_model = condor_train, 
                              label = "label_pred")
```

The predicted labels are saved in `condor_test$clustering$label_pred`.

```{r}
condor_test$clustering$label_pred[1:5,]
```

# Visualize the results

We provide here some costom code to overlap in a single plot the results from the train and test `condor` object. Nevertheless the independent results of each dataset can be vidualized with `cyCONDOR` built-in functions.

## Prepare the dataframe
```{r}
train <- as.data.frame(cbind(condor_train$umap$expr_orig, Phenograph = condor_train$clustering$phenograph_expr_orig_k_150[, 1]))

train$type <- "train"

test <- cbind(condor_test$umap$pred_expr_orig, 
              condor_test$clustering$label_pred)

test$Description <- NULL
test$Description <- NULL

colnames(test) <- c("UMAP1", "UMAP2", "Phenograph")

test$type <- "test"

vis_data <- rbind(train, test)
```

# Overlap UMAP
```{r}
vis_data$type <- factor(vis_data$type, levels = c("train", "test"))
```

```{r}
ggplot(data = vis_data, aes(x = UMAP1, y = UMAP2, color = type, alpha = type, size = type)) +
  geom_point() +
  scale_color_manual(values = c("gray", "#92278F")) +
  scale_alpha_manual(values = c(0.5, 1)) +
  scale_size_manual(values = c(0.1, 0.5)) +
  theme_bw() +
  theme(aspect.ratio = 1, panel.grid = element_blank()) + 
  ggtitle("UMAP projected")
```

```{r}
ggplot(data = vis_data, aes(x = UMAP1, y = UMAP2, color = Phenograph, alpha = type, size = type)) +
  geom_point() +
  scale_alpha_manual(values = c(0.1, 1)) +
  scale_size_manual(values = c(0.1, 0.5)) +
  theme_bw() +
  theme(aspect.ratio = 1, panel.grid = element_blank()) +
  ggtitle("Predicted cluster") + facet_wrap(~type)
```

# Session Info
```{r}
info <- sessionInfo()

info
```

