---
title: "Lexicography with R"
author: "Martin Schweinberger"
date: "`r format(Sys.time(), '%Y-%m-%d')`"
output:
  bookdown::html_document2: default
bibliography: bibliography.bib
link-citations: yes
---

```{r uq1, echo=F, fig.cap="", message=FALSE, warning=FALSE, out.width='100%'}
knitr::include_graphics("https://slcladal.github.io/images/uq1.jpg")
```

# Introduction{-}

This tutorial introduces lexicography with R and shows how to use R to create dictionaries and find synonyms through determining semantic similarity in R. While the initial example focuses on English, subsequent sections show how easily this approach can be generalized to languages other than English (e.g. German, French, Spanish, Italian, or Dutch). The entire R-markdown document for the sections below can be downloaded [here](https://slcladal.github.io/content/lex.Rmd).

Traditionally, dictionaries are listing of words that are commonly arranged alphabetically, which may include information on definitions, usage, etymologies, pronunciations, translation, etc. [see @agnes2002webster; @steiner1985dictionaries]. If such dictionaries, that are typically published as books contain translations of words in other languages, they are referred to as lexicons. Therefore, lexicographical references show the inter-relationships among lexical data, i.e. words.

Similarly, in computational linguistics, dictionaries represent a specific format of data where elements are linked to or paired with other elements in a  systematic way. *Computational lexicology* refers to a branch of computational linguistics, which is concerned with the use of computers in the study of lexicons. Hence, computational lexicology has been defined as the use of computers in the study of machine-readable dictionaries  [see e.g. @amsler1981structure]. Computational lexicology is distinguished from *computational lexicography*, which can be defined as the use of computers in the construction of dictionaries which is the focus of this tutorial. It should be noted, thought, that computational lexicology and computational lexicography are often used synonymously. 

## Preparation and session set up{-}

This tutorial is based on R. If you have not installed R or are new to it, you will find an introduction to and more information how to use R [here](https://slcladal.github.io/intror.html). For this tutorials, we need to install certain *packages* from an R *library* so that the scripts shown below are executed without errors. Before turning to the code below, please install the packages by running the code below this paragraph. If you have already installed the packages mentioned below, then you can skip ahead and ignore this section. To install the necessary packages, simply run the following code - it may take some time (between 1 and 5 minutes to install all of the packages so you do not need to worry if it takes some time).

```{r prep1, echo=T, eval = F, message=FALSE, warning=FALSE}
# set options
options(stringsAsFactors = F)         # no automatic data transformation
options("scipen" = 100, "digits" = 4) # suppress math annotation
# install packages
install.packages("dplyr")
install.packages("stringr")
install.packages("udpipe")
install.packages("tidytext")
install.packages("coop")
install.packages("cluster")
install.packages("flextable")
install.packages("textdata")
# install klippy for copy-to-clipboard button in code chunks
install.packages("remotes")
remotes::install_github("rlesur/klippy")
```

In a next step, we load the packages.


```{r prep2, echo=T, message=FALSE, warning=FALSE}
# load packages
library(dplyr)
library(stringr)
library(udpipe)
library(tidytext)
library(coop)
library(cluster)
library(flextable)
# activate klippy for copy-to-clipboard button
klippy::klippy()
```


Once you have installed R and RStudio and once you have initiated the session by executing the code shown above, you are good to go.

# Creating dictionaries

In a first step, we load a text. In this case, we load George Orwell's *Nineteen Eighty-Four*.

```{r ld, message=FALSE, warning=FALSE}
text <- readLines("https://slcladal.github.io/data/orwell.txt") %>%
  paste0(collapse = " ")
# show the first 500 characters of the text
substr(text, start=1, stop=500)
```

Next, we download a `udpipe` language model. In this case, we download a `udpipe` language model for English, but you can download `udpipe`  language models for more than 60 languages.


```{r lex1, eval = F, message=FALSE, warning=FALSE}
# download language model
m_eng   <- udpipe::udpipe_download_model(language = "english-ewt")
```

In my case, I have stored this model in a folder called `udpipemodels` and you can load it (if you have also save the model in a folder called `udpipemodels` within your Rproj folder as shown below.
)
```{r lex2, message=FALSE, warning=FALSE}
# load language model from your computer after you have downloaded it once
m_eng <- udpipe_load_model(file = here::here("udpipemodels", "english-ewt-ud-2.5-191206.udpipe"))
```

In a next step, we implement the part-of-speech tagger.

```{r lex3, message=FALSE, warning=FALSE}
# tokenise, tag, dependency parsing
text_ann <- udpipe::udpipe_annotate(m_eng, x = text) %>%
  # convert into a data frame
  as.data.frame() %>%
  # remove columns we do not need
  dplyr::select(-sentence, -paragraph_id, -sentence_id, -feats, 
                -head_token_id, -dep_rel, -deps, -misc)
# inspect
head(text_ann, 10)
```


We can now use the resulting table to generate a first, basic dictionary that holds information about the word form (*token*), the part-of speech tag (*upos*), the lemmatized word type (*lemma*), and the frequency with which the word form is used as that part-of speech.

```{r lex7, message=FALSE, warning=FALSE}
# generate dictionary
text_dict_raw <- text_ann %>%
  # remove non-words
  dplyr::filter(!stringr::str_detect(token, "\\W")) %>%
  # filter out numbers
  dplyr::filter(!stringr::str_detect(token, "[0-9]")) %>%
  # group data
  dplyr::group_by(token, lemma, upos) %>%
  # summarize data
  dplyr::summarise(frequency = dplyr::n()) %>%
  # arrange by frequency
  dplyr::arrange(-frequency)
# inspect
head(text_dict_raw, 10)
```

The above display is ordered by frequency but it is, of course more common, to arrange dictionaries alphabetically. To do this, we can simply use the `àrrange` function from the `dplyr` package as shown below.


```{r lex8, message=FALSE, warning=FALSE}
# generate dictionary
text_dict <- text_dict_raw %>%
  # arrange alphabetically
  dplyr::arrange(token)
# inspect
head(text_dict, 10)
```

We have now generated a basic dictionary of English but, as you can see above, there are still some errors as the part-of-speech tagging was not perfect. As such, you will still need to check and edit the results manually but you have already a rather clean dictionary based on George Orwell's *Nineteen Eighty-Four* to work with. 


## Correcting and Extending Dictionaries{-}

Fortunately, it is very easy in R to correct entries, i.e., changing lemmas or part-of-speech tags, and to extend entries, i.e., adding additional layers of information such as urls or examples. 

We will begin to extend our dictionary by adding an additional column (called `annotation`) in which we will add information.


```{r ext1, message=FALSE, warning=FALSE}
# generate dictionary
text_dict_ext <- text_dict %>%
  # removing an entry
  dplyr::filter(!(lemma == "a" & upos == "NOUN")) %>%
  # editing entries
  dplyr::mutate(upos = ifelse(lemma == "aback" & upos == "NOUN", "PREP", upos)) %>%
  # adding comments 
  dplyr::mutate(comment = dplyr::case_when(lemma == "a" ~ "also an before vowels",
                                           lemma == "Aaronson" ~ "Name of someone.", 
                                           T ~ ""))
# inspect
head(text_dict_ext, 10)
```

To make it a bit more interesting but also keep this tutorial simple and straight-forward, we will add information about the polarity and emotionally of the words in our dictionary. We can do this by performing a sentiment analysis on the lemmas using the `tidytext` package.

The `tidytext` package contains three sentiment dictionaries (`nrc`, `bing`, and `afinn`). For the present purpose, we use the `ncr`dictionary which represents the Word-Emotion Association Lexicon [@mohammad2013crowdsourcing]. The Word-Emotion Association Lexicon which comprises 10,170 terms, and in which lexical elements are assigned scores based on ratings gathered through the crowd-sourced Amazon Mechanical Turk service. For the Word-Emotion Association Lexicon raters were asked whether a given word was associated with one of eight emotions. The resulting associations between terms and emotions are based on 38,726 ratings from 2,216 raters who answered a sequence of questions for each word which were then fed into the emotion association rating [cf. @mohammad2013crowdsourcing]. Each term was rated 5 times. For 85 percent of words, at least 4 raters provided identical ratings. For instance, the word *cry* or *tragedy* are more readily associated with SADNESS while words such as *happy* or *beautiful* are indicative of JOY and words like *fit* or *burst* may indicate ANGER. This means that the sentiment analysis here allows us to investigate the expression of certain core emotions rather than merely classifying statements along the lines of a crude positive-negative distinction.

To be able to use the Word-Emotion Association Lexicon we need to add another column to our data frame called `word` which simply contains the lemmatized word. The reason is that the lexicon expects this column and only works if it finds a word column in the data. The code below shows how to add the emotion and polarity entries to our dictionary.


```{r ext3, message=FALSE, warning=FALSE}
# generate dictionary
text_dict_snt <- text_dict_ext %>%
  dplyr::mutate(word = lemma) %>%
  dplyr::left_join(get_sentiments("nrc")) %>%
  dplyr::group_by(token, lemma, upos, comment) %>%
  dplyr::summarise(sentiment = paste0(sentiment, collapse = ", "))
# inspect
head(text_dict_snt, 10) 
```

The resulting extended dictionary now contains not only the token, the lemma, and the pos-tag but also the sentiment from the Word-Emotion Association Lexicon.

## Generating dictionaries for other languages{-}

As mentioned above, the procedure for generating dictionaries can easily be applied to languages other than English. If you want to follow exactly the procedure described above, then the language set of the TreeTagger is the limiting factors as its R implementation only supports English, German, French, Italian, Spanish, and Dutch. fa part-of-speech tagged text in another language is already available to you, and you do not require the TreeTagger for the part-of-speech tagging, then you can skip the code chunk that is related to the tagging and you can modify the procedure described above to virtually any language.

We will now briefly create a German dictionary based on a subsection of the fairy tales collected by the brothers Grimm to show how the above procedure can be applied to a language other than English. In a  first step, we load a German text into R. 


```{r none1, message=FALSE, warning=FALSE}
grimm <- readLines("https://slcladal.github.io/data/GrimmsFairytales.txt",
                   encoding = "latin1") %>%
  paste0(collapse = " ")
# show the first 500 characters of the text
substr(grimm, start=1, stop=200)
```


Next, we download a `udpipe` language model. In this case, we download a `udpipe` language model for German, but you can download `udpipe`  language models for more than 60 languages.


```{r none2, eval = F, message=FALSE, warning=FALSE}
# download language model
udpipe::udpipe_download_model(language = "german-hdt")
```

In my case, I have stored this model in a folder called `udpipemodels` and you can load it (if you have also save the model in a folder called `udpipemodels` within your Rproj folder as shown below).

```{r none3, message=FALSE, warning=FALSE}
# load language model from your computer after you have downloaded it once
m_ger <- udpipe_load_model(file = here::here("udpipemodels",
                                             #"german-hdt-ud-2.5-191206.udpipe"))
                                             "german-gsd-ud-2.5-191206.udpipe"))
```

In a next step, we generating the dictionary based on the brothers' Grimm fairy tales. We go through the same steps as for the English dictionary and collapse all the steps into a single code block. 

```{r none4, message=FALSE, warning=FALSE}
# tokenise, tag, dependency parsing
grimm_ann <- udpipe::udpipe_annotate(m_ger, x = grimm) %>%
  # convert into a data frame
  as.data.frame() %>%
  # remove non-words
  dplyr::filter(!stringr::str_detect(token, "\\W")) %>%
  # filter out numbers
  dplyr::filter(!stringr::str_detect(token, "[0-9]")) %>%
  dplyr::group_by(token, lemma, upos) %>%
  dplyr::summarise(frequency = dplyr::n()) %>%
  dplyr::arrange(lemma)
# inspect
head(grimm_ann, 10)
```





As with the English dictionary, we have created a customized German dictionary based of a subsample of the brothers' Grimm fairy tales holding the word form(*token*), the part-of-speech tag (*tag*), the lemmatized word type (*lemma*), the general word class (*wclass*), ad the frequency with which a word form occurs as a part-of-speech in the data (*frequency*).

# Finding synonyms: creating a thesaurus

Another task that is quite common in lexicography is to determine if words share some form of relationship such as whether they are synonyms or antonyms. In computational linguistics, this is commonly determined based on the collocational profiles of words. These collocational profiles are also called *word vectors* or *word embeddings* and approaches which determine semantic similarity based on collocational profiles or word embeddings are called distributional approaches (or distributional semantics). The basic assumption of distributional approaches is that words that occur in the same context and therefore have similar collocational profiles are also semantically similar. In fact, various packages, such as `qdap` or , `wordnet` already provide synonyms for terms (all of which are based on similar collocational profiles) but we would like to determine if words are similar without knowing it in advance. 

In this example, we want to determine if two degree adverbs (such as *very*, *really*, *so*, *completely*, *totally*, *amazingly*, etc.) are synonymous and can therefore be exchanged without changing the meaning of the sentence (or, at least, not changing it dramatically). This is relevant in lexicography as such terms can then be linked to each other and inform readers that these words are interchangeable. 

As a first step, we load the data which contains three columns: 

* one column holding the degree adverbs which is called *pint* 

* one column called *adjs* holding the adjectives that the degree adverbs have modified

* one column called *remove* which contains the word *keep* and which we will remove as it is not relevant for this tutorial

When loading the data, we 

* remove the *remove* column 

* rename the *pint* column as *degree_adverb* 

* rename the *adjs* column as *adjectives*

* filter out all instances where the degree adverb column has the value `0` (which means that the adjective was not modified)

* remove instances where *well* functions as a degree adverb (because it behaves rather differently from other degree adverbs)
 
```{r syn1, message=FALSE, warning=FALSE}
# load data
degree_adverbs <- base::readRDS(url("https://slcladal.github.io/data/dad.rda", "rb")) %>%
  dplyr::select(-remove) %>%
  dplyr::rename(degree_adverb = pint,
                adjective = adjs) %>%
  dplyr::filter(degree_adverb != "0",
                degree_adverb != "well")
# inspect
head(degree_adverbs, 10)
```



In a next step, we create a matrix from this data frame which maps how often a given amplifier co-occurred with a given adjective. In text mining, this format is called a text-document matrix or tdm (which is a transposed [document-term matrix](https://en.wikipedia.org/wiki/Document-term_matrix) of dtm).

```{r vsm3}
# tabulate data (create term-document matrix)
tdm <- ftable(degree_adverbs$adjective, degree_adverbs$degree_adverb)
# extract amplifiers and adjectives 
amplifiers <- as.vector(unlist(attr(tdm, "col.vars")[1]))
adjectives <- as.vector(unlist(attr(tdm, "row.vars")[1]))
# attach row and column names to tdm
rownames(tdm) <- adjectives
colnames(tdm) <- amplifiers
# inspect data
tdm[1:5, 1:5]
```

In a next step, we extract the expected values of the co-occurrences if the amplifiers were distributed homogeneously and calculate the *Pointwise Mutual Information* (PMI) score and use that to then calculate the *Positive Pointwise Mutual Information* (PPMI) scores. According to @levshina2015linguistics 327 - referring to @bullinaria2007extracting - PPMI perform better than PMI as negative values are replaced with zeros. In a next step, we calculate the cosine similarity which will for the bases for the subsequent clustering.

```{r vsm5, message=F, warning=F}
# compute expected values
tdm.exp <- chisq.test(tdm)$expected
# calculate PMI and PPMI
PMI <- log2(tdm/tdm.exp)
PPMI <- ifelse(PMI < 0, 0, PMI)
# calculate cosine similarity
cosinesimilarity <- cosine(PPMI)
# inspect cosine values
cosinesimilarity[1:5, 1:5]
```

As we have now obtained a similarity measure, we can go ahead and perform a cluster analysis on these similarity values. However, as we have to extract the maximum values in the similarity matrix that is not 1 as we will use this to create a distance matrix. While we could also have simply subtracted the cosine similarity values from 1 to convert the similarity matrix into a distance matrix, we follow the procedure proposed by @levshina2015linguistics.

```{r vsm6, message=FALSE, warning=FALSE}
# find max value that is not 1
cosinesimilarity.test <- apply(cosinesimilarity, 1, function(x){
  x <- ifelse(x == 1, 0, x) } )
maxval <- max(cosinesimilarity.test)
# create distance matrix
amplifier.dist <- 1 - (cosinesimilarity/maxval)
clustd <- as.dist(amplifier.dist)
```

In a next step, we visualize the results of the semantic vector space model as a dendrogram.

```{r vsm8, message=FALSE, warning=FALSE}
# create cluster object
cd <- hclust(clustd, method="ward.D")    
# plot cluster object
plot(cd, main = "", sub = "", yaxt = "n", ylab = "", xlab = "", cex = .8)
```

The clustering solution shows that, as expected, *completely*, *extremely*, and *totally* - while similar to each other and thus interchangeable with each other - form a separate cluster from all other amplifiers. In addition, *real* and *really* form a cluster together. The clustering of *very*, *pretty*, *so*, *really*, and *real* suggest that these amplifiers are more or less interchangeable with each other but not with *totally*, *completely*, and *extremely*. 

To extract synonyms automatically, we can use the cosine similarity matrix that a´we generated before. This is what we need to do:

* generate a column called word
* replace the perfect similarity values of the diagonal with 0
* look up the lowest value, i.e. the word that has the lowest distance to a given word
* create a vector which holds those words (the synonym candidates).

```{r vsm9, message=FALSE, warning=FALSE}
syntb <- cosinesimilarity %>%
  as.data.frame() %>%
  dplyr::mutate(word = colnames(cosinesimilarity)) %>%
  dplyr::mutate_each(funs(replace(., . == 1, 0))) %>%
  dplyr::mutate(synonym = colnames(.)[apply(.,1,which.max)]) %>%
  dplyr::select(word, synonym)
syntb
```

***

<div class="warning" style='padding:0.1em; background-color:#51247a; color:#f2f2f2'>
<span>
<p style='margin-top:1em; text-align:center'>
<b>NOTE</b><br><br>Remember that this is only a tutorial! A proper study would have to take the syntactic context into account because, while we can say *This really great tutorial helped me a lot*. we probably would not say *This so great tutorial helped me a lot*. This is because so syntactically more restricted and is strongly disfavored in attributive contexts. Therefore, the syntactic context would have to be considered in a more thorough study.</p>
<p style='margin-left:1em;'>
</p></span>
</div>

<div class="question">` 


</div>`

***

There are many more useful methods for identifying semantic similarity. A very useful method (which we have implemented here but only superficially is Semantic Vector Space Modeling. If you want to know more about this, this [tutorial by Gede Primahadi Wijaya Rajeg, Karlina Denistia, and Simon Musgrave](https://gederajeg.github.io/vector_space_model_indonesian/) [@rajeg2020semvec] is highly recommended and will give a better understanding of SVM but this should suffice to get you started.

# Creating bilingual dictionaries

Dictionaries commonly contain information about elements. Bilingual or translation dictionaries represent a sub-category of dictionaries that provide a specific type of information about a given word: the translation of that word in another language. In principle, generating translation dictionaries is relatively easy and straight forward. However, not only is the devil hidden in the details but the generation of data-driven translation dictionaries also require a substantial data set consisting of sentences and their translation. This is often quite tricky as well aligned translations are unfortunately, and unexpectedly, rather hard to come by. 

Despite these issues, if you have access to clean and well aligned, parallel multilingual data, then you simply need to check which correlation between the word in language A and language B is the highest and you have a likely candidate for its translation. The same procedure can be extended to generate multilingual dictionaries. Problems arise due to grammatical differences between languages, idiomatic expressions, homonymy and polysemy as well as due to word class issues. The latter, word class issues, can be solved by part-of-speech tagging and then only considering words that belong to the same (or realistically similar) parts-of speech. The other issues can also be solved but require substantial amounts of (annotated) data.   

To explore how to generate a multilingual lexicon, we load a sample of English sentences and their German translations.

```{r trans1}
# load translations
translations <- readLines("https://slcladal.github.io/data/translation.txt",
                          encoding = "UTF-8", skipNul = T)
```

```{r trans1b, echo = F, message=FALSE, warning=FALSE}
translations %>%
  as.data.frame() %>%
  head(15) %>%
  flextable() %>%
  flextable::set_table_properties(width = .5, layout = "autofit") %>%
  flextable::theme_zebra() %>%
  flextable::fontsize(size = 12) %>%
  flextable::fontsize(size = 12, part = "header") %>%
  flextable::align_text_col(align = "center") %>%
  flextable::set_caption(caption = "First 15 rows of the translations data.")  %>%
  flextable::border_outer()
```


In a next step, we generate separate tables which hold the German and English sentences. However, the sentences and their translations are identified by an identification number (*id*) so that we keep the information about which sentence is linked to which translation.


```{r trans2}
# german sentences
german <- str_remove_all(translations, " — .*") %>%
  str_remove_all(., "[:punct:]")
# english sentences
english <- str_remove_all(translations, ".* — ") %>%
  str_remove_all(., "[:punct:]")
# sentence id
sentence <- 1:length(german)
# combine into table
germantb <- data.frame(sentence, german)
# sentence id
sentence <- 1:length(english)
# combine into table
englishtb <- data.frame(sentence, english)
```

```{r trans2b, echo = F, message=FALSE, warning=FALSE}
germantb %>%
  as.data.frame() %>%
  head(15) %>%
  flextable() %>%
  flextable::set_table_properties(width = .5, layout = "autofit") %>%
  flextable::theme_zebra() %>%
  flextable::fontsize(size = 12) %>%
  flextable::fontsize(size = 12, part = "header") %>%
  flextable::align_text_col(align = "center") %>%
  flextable::set_caption(caption = "First 15 rows of the germantb data.")  %>%
  flextable::border_outer()
```

We now unnest the tokens (split the sentences into words) and subsequently add the translations which we again unnest. The resulting table consists of two columns holding German and English words. The relevant point here is that each German word is linked with each English word that occurs in the translated sentence.


```{r trans3, warning = F, message=F}
library(plyr)
# tokenize by sentence: german
transtb <- germantb %>%
  unnest_tokens(word, german) %>%
  # add english data
  plyr::join(., englishtb, by = "sentence") %>%
  unnest_tokens(trans, english) %>%
  dplyr::rename(german = word,
                english = trans) %>%
  dplyr::select(german, english) %>%
  dplyr::mutate(german = factor(german),
                english = factor(english))
```

```{r trans3b, echo = F, message=FALSE, warning=FALSE}
transtb %>%
  as.data.frame() %>%
  head(15) %>%
  flextable() %>%
  flextable::set_table_properties(width = .25, layout = "autofit") %>%
  flextable::theme_zebra() %>%
  flextable::fontsize(size = 12) %>%
  flextable::fontsize(size = 12, part = "header") %>%
  flextable::align_text_col(align = "center") %>%
  flextable::set_caption(caption = "First 15 rows of the transtb data.")  %>%
  flextable::border_outer()
```

Based on this table, we can now generate a term-document matrix which shows how frequently each word co-occurred in the translation of any of the sentences. For instance, the German word *alles* occurred one time in a translation of a sentence which contained the English word *all*.

```{r trans4}
# tabulate data (create term-document matrix)
tdm <- ftable(transtb$german, transtb$english)
# extract amplifiers and adjectives 
german <- as.vector(unlist(attr(tdm, "col.vars")[1]))
english <- as.vector(unlist(attr(tdm, "row.vars")[1]))
# attach row and column names to tdm
rownames(tdm) <- english
colnames(tdm) <- german
# inspect data
tdm[1:10, 1:10]
```

Now, we reformat this co-occurrence matrix so that we have the frequency information that is necessary for setting up 2x2 contingency tables which we will use to calculate the co-occurrence strength between each word and its potential translation. 


```{r trans5}
coocdf <- as.data.frame(as.matrix(tdm))
cooctb <- coocdf %>%
  dplyr::mutate(German = rownames(coocdf)) %>%
  tidyr::gather(English, TermCoocFreq,
                colnames(coocdf)[1]:colnames(coocdf)[ncol(coocdf)]) %>%
  dplyr::mutate(German = factor(German),
                English = factor(English)) %>%
  dplyr::mutate(AllFreq = sum(TermCoocFreq)) %>%
  dplyr::group_by(German) %>%
  dplyr::mutate(TermFreq = sum(TermCoocFreq)) %>%
  dplyr::ungroup(German) %>%
  dplyr::group_by(English) %>%
  dplyr::mutate(CoocFreq = sum(TermCoocFreq)) %>%
  dplyr::arrange(German) %>%
  dplyr::mutate(a = TermCoocFreq,
                b = TermFreq - a,
                c = CoocFreq - a, 
                d = AllFreq - (a + b + c)) %>%
  dplyr::mutate(NRows = nrow(coocdf))%>%
  dplyr::filter(TermCoocFreq > 0)
```


```{r trans5b, echo = F, message=FALSE, warning=FALSE}
cooctb %>%
  as.data.frame() %>%
  head(15) %>%
  flextable() %>%
  flextable::set_table_properties(width = .75, layout = "autofit") %>%
  flextable::theme_zebra() %>%
  flextable::fontsize(size = 12) %>%
  flextable::fontsize(size = 12, part = "header") %>%
  flextable::align_text_col(align = "center") %>%
  flextable::set_caption(caption = "First 15 rows of the cooctb data.")  %>%
  flextable::border_outer()
```

In a final step, we extract those potential translations that correlate most strongly with each given term. The results then form a list of words and their most likely translation. 

```{r trans6, warning = F, message=F}
translationtb <- cooctb  %>%
  dplyr::rowwise() %>%
  dplyr::mutate(p = round(as.vector(unlist(fisher.test(matrix(c(a, b, c, d), ncol = 2, byrow = T))[1])), 5), 
                x2 = round(as.vector(unlist(chisq.test(matrix(c(a, b, c, d), ncol = 2, byrow = T))[1])), 3)) %>%
  dplyr::mutate(phi = round(sqrt((x2/(a + b + c + d))), 3),
                expected = as.vector(unlist(chisq.test(matrix(c(a, b, c, d), ncol = 2, byrow = T))$expected[1]))) %>%
  dplyr::filter(TermCoocFreq > expected) %>%
  dplyr::arrange(-phi) %>%
  dplyr::select(-AllFreq, -a, -b, -c, -d, -NRows, -expected)
```

```{r trans6b, echo = F, message=FALSE, warning=FALSE}
translationtb %>%
  as.data.frame() %>%
  head(15) %>%
  flextable() %>%
  flextable::set_table_properties(width = .75, layout = "autofit") %>%
  flextable::theme_zebra() %>%
  flextable::fontsize(size = 12) %>%
  flextable::fontsize(size = 12, part = "header") %>%
  flextable::align_text_col(align = "center") %>%
  flextable::set_caption(caption = "First 15 rows of the translationtb data.")  %>%
  flextable::border_outer()
```

The results show that even using the very limited data base can produce some very reasonable results. In fact, based on the data that we used here, the first translations appear to be very sensible, but the mismatches also show that more data is required to disambiguate potential translations.

While this method still requires manual correction, it is a very handy and useful tool for generating custom bilingual dictionaries that can be extended to any set of languages as long as these languages can be represented as distinct words and as long as parallel data is available. 


# Going further: crowd-sourced dictionaries with R and Git

While it would go beyond the scope of this tutorial, it should be noted that the approach for creating dictionaries can be applied to crowed-sourced dictionaries. To do this, you could, e.g. upload your dictionary to a Git repository such as [GitHub](https://github.com/) or [GitLab](https://about.gitlab.com/) which would then allow everybody with an account on either of these platforms to add content to the dictionary. 

```{r git1, echo=FALSE, out.width= "50%", out.extra='style="float:right; padding:15px"'}
knitr::include_graphics("https://slcladal.github.io/images/git.png")
```

To add to the dictionary, contributors would simply have to fork the repository of the dictionary and then merge with the existing, original dictionary repository. The quality of the data would meanwhile remain under control of the owner of the original repository he they can decide on a case-by-case basis which change they would like to accept. In addition, and because Git is a version control environment, the owner could also go back to previous versions, if they think they erroneously accepted a change (merge).

This option is particularly interesting for the approach to creating dictionaries presented here because R Studio has an integrated and very easy to use pipeline to Git (see, e.g.,  [here](https://support.rstudio.com/hc/en-us/articles/200532077-Version-Control-with-Git-and-SVN) and [here](https://happygitwithr.com/rstudio-git-github.html))

We have reached the end of this tutorial and you now know how to create and modify networks in R and how you can highlight aspects of your data. 

<br><br>

# Citation & Session Info {-}

Schweinberger, Martin. `r format(Sys.time(), '%Y')`. *Lexicography with R*. Brisbane: The University of Queensland. url: https://slcladal.github.io/lex.html  (Version `r format(Sys.time(), '%Y.%m.%d')`).

```
@manual{schweinberger`r format(Sys.time(), '%Y')`lex,
  author = {Schweinberger, Martin},
  title = {Lexicography with R},
  note = {https://slcladal.github.io/lex.html},
  year = {`r format(Sys.time(), '%Y')`},
  organization = "The University of Queensland, Australia. School of Languages and Cultures},
  address = {Brisbane},
  edition = {`r format(Sys.time(), '%Y.%m.%d')`}
}
```

```{r fin}
sessionInfo()
```




# References{-}

***

[Back to top](#introduction)

[Back to HOME](https://slcladal.github.io/index.html)

***

