In science, at least the way I understand it, we want to systematically and empirically acquire knowledge about the natural world. This encompasses the formulation and testing of general laws through the scientific method. In biology, we want to extract knowledge about life from data. How can we do that?
We record data by observing and measuring the world and life around us, which transform into information once we assign labels or meanings to them, Figure 1. As a means to distill information from data, which may eventually lead to the gain of knowledge, humanity has discovered the need for statistics and philosophy. It arises from uncertainty and variability being pervasive. Statistics serves as the compass that guides us through the labyrinth of data, enabling us to derive meaningful insights and make informed decisions. In this thesis, the main contributions are efforts to organize, summarize, and interpret a deluge of data that is recently flooding biology. However, data -- no matter what and how big -- can always only provide a glimpse of information about a potentially complex system. Therefore, even if we are dealing with a flood, we will explore throughout this thesis what we can and cannot learn from limited bits of information and, how problems arise that statistics can or cannot solve. Just like any good tool, it can only ever help us uncover information that is actually present in the data. If information is not present, we face a problem of 'Garbage in, garbage out'.
Figure 1: Data are the record of observations, which turn into information once we attach a label or meaning to it. There may be labeled data, however, with no information in it. In information theory, the mathematical study of information and communication, we would say it has a low entropy, meaning there is no surprise for us in the recordings. If we get surprised by information in data, it may teach us something new, and contribute to knowledge, once we find the connections and causalities causing the surprise in the data. Learning something new can happen at any point, maybe we record some more observations which give us new insights (that surprise us), from which we can deduce new knowledge that we can weave into our current beliefs. In contrast, we can also ignore all of these prior steps, methodology and philosophy and transform data, information, knowledge, and insights into a conspiracy. Alternatively, we can choose to set aside rationality and engage with our emotions, creating something profoundly meaningful from any of the given pieces -- art. Artistic expression allows us to explore the emotional and subjective dimensions of our experiences, highlighting the diverse ways we can interpret and interact with the world around us.
Measuring things can be hard. This may be because of its unstable nature (How many hours of sunshine do we get in a day in Norwich?), or because measuring it is a technically challenging (How fast can flies fly?). We still want to try recording observations, of course, because we are curious, and then we can see whether there is information in our data, that we can potentially deduce knowledge from. To achieve this, we want to measure many times to capture variability and potential measuring errors. Unfortunately, we can do our best to measure in the most excellent way possible, and still end up with limitations (in terms of information, entropy, surprise) of our data.
We can use Probability theory to express the uncertainty of a measurement. Pioneers Thomas Bayes and Pierre-Simon Laplace explored this in their work in the 18th and 19th centuries. As the story goes, their contributions were largely discredited and forgotten. Until many many decades later several researchers, among them Harold Jeffreys, Edwin Thompson Jaynes, and Richard Cox rediscovered and popularised the powers of Probability theory in statistics. They founded what we now know as Bayesian statistics.
Everything may have started with another mathematician, James Bernoulli, asking the question of how to reason in situations where one cannot argue with certainty. Bernoulli formulated a difference between deductive logic and inductive logic. Deductive reasoning moves from general principles to specific instances, and inductive reasoning moves from specific instances to general principles ([Sivia & Skilling, 2006](http:/ / books.google.com/ books?id=lYMSDAAAQBAJ)). He was pondering over the question whether we can use the mathematics and mechanisms of probability, which can be used in games of chance, for inference problems appearing in our everyday lives ...
It is established now that we can use probability theory for inference problems and in the next pages I will try to introduce how 'Bayesian inference' works, before we will encounter new success stories later in the thesis. To the early pioneers, a probability represented a degree of belief or plausibility of an event: How much do we think something is true, given some evidence we have at hand? This is the essence of hypothesis testing and model comparison we are going to use.
You are playing a simple card game with a full deck of cards. Your partner reveals to you the the top card on the stack, now it is your turn to guess whether the next card in the stack has a higher or lower value. It's the beginning of the game, they show you a 2. Do you choose 'above' or 'below', and why?
You are rushing to a meeting in a university building you have not been in before. Suddenly, you feel your period starting. You have no female hygiene products on you. You need to find another woman to hopefully help you out. There is a junction in your path, one leading to the computer science department, one leading to the department for environmental sciences. Where do you go and why?
Chances are good, you have just intuitively done what took humanity a while to figure out consciously. Your past experiences in playing cards and navigating university buildings have informed your decision on your best choice of action. You naturally evaluated which choice is more likely to bring you success. Now, how would your decisions change, however, if you are at the end of the card game, there are 4 cards left and you have not yet seen a single 1? How would your decisions change if it is 2024 and you would assume university staff has a good representation of all genders no matter the discipline?
You can answer these questions easily because you have been confronted with them a lot of times in your life and it comes intuitive to you now. Let's pretend for a minute you are a child and you have never played a card game in your life before. What do you do? You play, and lose and win and lose until you eventually start noticing patterns. You start to get a feeling for the chances of winning with certain values. The pioneers I mentioned earlier turned this intuition into a mathematical framework: probability theory. What's the probability of winning the game by saying 'above' given your partner is showing you a 1? 2? 3? 4? 5? And so on? And furthermore, how do we learn to get better at this game by playing it lots of times?
Let's say,
Figure 2: The blue event
We are interested in the intersection of these events, the conditional probability of
The probabilities
From here we can get the intersection, the probability of
The card that is shown to us, is nothing other than some data
which we call posterior probability, the probability that our hypothesis
or more precisely,
given that everything is dependent on the hypothesis that we formulate.
As I stated earlier, the goal in science is to deduce knowledge from data. We want to find the general principles explaining specific observations we make. So we formulate hypotheses, that we wish to test by making observations and evaluating whether they can explain what we are seeing. A hypothesis is a specific statement or proposition about the system we are investigating and we can only test it, by also defining a model. A model in Bayesian statistics refers to a mathematical representation of the data-generating process: How did the data come about? It includes the likelihood function, which describes how the observed data is generated given certain parameters. The model incorporates prior beliefs about the parameters through the prior distribution. A hypothesis, the statement or proposition we are making, is about a parameter or a set of parameters within the context of the model. We can test hypotheses by calculating the posterior probabilities of these hypotheses given the observed data. The likelihood
The denominator normalises our posterior probability. It is the probability of the data. This normalising factor is also called the evidence (or the marginal likelihood), and it got this prestigious name because it will later play an important role in hypothesis testing and Bayes factors.
Probably, this is not how you have learned to play card games. You did not think about posterior probabilities, hypotheses and models. What you did is thousands of experiments, collecting experiences of failing and learning until you reached your current knowledge about card games. This learning process over the years of your life, is a knowledge updating process that is essentially the same as learning from data in our scientific experiments, follow me through Figures 3, 4, and 5.
Figure 3: If we are playing a card game for the first time in our life we have no prior knowledge how to optimise our chances for winning. We have no tactics. We just start an experiment (Experiment 1), collect some experience of winning and losing, data (Data 1), and eventually know more about how to win afterwards (posterior knowledge). If we are playing as an adult, we have some experience in card games. This prior knowledge can help us to understanding the game before we have even started playing (collecting new data).
The posterior is a probability distribution describing our current knowledge. For example, the probability that we are going to win the card game, given our current knowledge about card games. The probability distribution over the event of winning
With every game we play, we learn -- we do another Bayesian knowledge updating step, just as described mathematically in our first equations for posteriors. The equations formalise how to learn more about the game by updating the probability distribution over
Figure 4: With every game we play and every experiment we conduct, we are updating our knowledge about card games, this world and how to find tampons in emergency situations. All Prior knowledge can be taken into account to update our posterior knowledge. As an adult, we have all of our past experiences behind us, making it easier to make an informed decision between 'above' and 'below'. A researcher with a lot of background knowledge will read and interpret a new paper in their field in very different light than a student immersing for the first time.
Figure 5: Another visualisation of the concept of Bayesian knowledge updating: This figure is exactly the same as the last ones but by applying colours we want to emphasise the learning process (starting from blue) and the mixing in of new information (shades of red) changing our current view, the posterior (purple). In this case the colours are getting more and more intense and vibrant. This is not the case in all learning processes, however. What if experiments show opposing results?
Because all of what we have talked about so far happens very fast and automatically, we do not think about it too much. Now comes the bad news: Humans tend to be bad at recognising which bits of prior knowledge are important to make good decisions. Science communicator Grant Sanderson identified the difficulty aptly, stating: 'Rationality is not about knowing facts, it's about recognising which facts are relevant' (see [Bayes theorem, the geometry of changing beliefs -- YouTube](https:/ / www.youtube.com/ watch?v=jFUhTmOSdGQ), and [The medical test paradox, and redesigning Bayes’ rule -- YouTube](https:/ / www.youtube.com/ watch?v=lG4VkPoG3ko))
For example, while we were focussing on finding a woman in this university building (especially in a stressful situation), did we consider how big the two departments are? How likely are we going to meet any person in the department? Are scientists going to be working from home? Are they going to be on fieldwork? Those are the moments where theory and practice deviate far from each other and where we are losing in card games because we under- or over-estimate our chances being distracted by irrelevant facts. Prior knowledge matters.
A famous example demonstrating how prior information can fundamentally change the interpretation of a situation is the Medical Test Paradox. It is not a real paradox but a veridical paradox teaching us how an accurate test (for a disease, or a hypothesis) is not necessarily a predictive test (for having the disease, or a hypothesis being true).
Assume there is a breast cancer screening and 1000 women are tested. 10 of those women do suffer from breast cancer, of which 9 get a positive test result. 89 further women get a positive test result, although they do not have cancer, just like the 901 cancer-free women with negative test results. You are one of the 1000 women in the test and you get your results back with a positive outcome, how likely is it, that you have cancer?
In your first shock, you might be sure you have cancer, because you consider the worst and you feel you have good evidence -- a positive test result -- that you suffer from the disease. But eventually you will question how accurate the test is that is turning your world upside down. Conventionally, we can calculate test accuracy statistics from the numbers given above, numbers that are important to be available for all medical tests for quality assurance. Table 1 will help to remind us how test statistics are usually calculated.
Table 1: There are 4 possible scenarios that we can count for the outcome of a test. The result can be positive (+) and negative (-) and the condition can be given (P) or absent (N). For a person with the disease (P) receiving a positive test (+) we get a true positive hit (TP), a person with the condition (P) receiving a negative test (-) a false negative hit (FN). Subsequently we get a false positive hit (FP) for a person without the disease (N) receiving a positive test result (+), and a true negative hit (TN) for a person correctly being identified as disease-free. We can summarise the performance of our (medical) test in four values: the true positive rate (TPR) also called Sensitivity of a test, the false negative rate (FNR), the false positive rate (FPR), and the true negative rate (TNR), also known as specificity.
Table 2: We are using a cancer screening as an example for the medical test paradox. We are testing a group of 1000 people, of which 10 have the condition (P) and 990 do not (N). The test is showing a positive result (+) on 98 patients and a negative result (-) on 902. 89 false positive (FP) cases and 1 false negative (FN) case have occurred.
The test in our example has a sensitivity of 90 %,
and a specificity of 91 %,
These numbers were given to a group of gynecologists in a study in the 90s ([Gigerenzer, 2002](http:/ / books.google.com/ books?id=KJ7nrlJqcRYC)), where they were asked the same question as above. What are the odds that your patient actually has cancer given they received a positive test result? They were given the answers (A) 9 in 10, (B) 8 in 10, (C) 1 in 10 and (D) 1 in 100; What do you think?
In this influential piece of research for the fields of psychology and medicine we have learned how people often misunderstand probabilities and the significance of base rates when interpreting medical test results. But often is actually an understatement: In this specific experiment we have learned that the group of healthcare professionals have performed worse than random in the test. Most picked the worst case (A), while (C) is the true answer. Once we see a visualisation of the test statistics, we get a better intuition for what is going on here: We underestimate what the chances to have cancer are, Figure 6. We can conclude that there is the need to find better ways to communicate medical tests, if their statistics are heavily misinterpreted. But now, if medical tests are so likely to be misinterpreted, what about scientific results?
Figure 6: The medical test example we are using is a cancer screening with 1000 participants represented in this figure by 1000 circles. There is a 1 % prevalence for the disease, the test has a sensitivity of 90 %, and a specificity of 91 %. If there are 10 cancer cases among 1000 participants, receiving one of 98 positive results does update your chances of having cancer from 10 in 1000 to 1 in 10.
The implications of the medical test paradox will follow us through the rest of this thesis, out into the world and our everyday life. How do we combat the misunderstanding of tests?
One good way to help us correctly estimate how likely we are to suffer from a disease given a positive test result is by rephrasing the question. By what factor have the odds of carrying a disease increased, given a positive test result, as compared to before the test? This factor is called a Bayes factor. Before we do the test we might know how widespread the disease is. We can update this prior knowledge about our chances of having the disease by doing a test. It depends on the accuracy of the test how much we learn from a positive test result. To summarise the test statistics we can calculate this Bayes factor, a single value that summarises the 4 values we had been given earlier (Table 1) with TP (True Positive), FP (False Positive), TN (True Negative), FN (False Negative); or the 2 in form of sensitivity and specificity).
If we want to express the posterior knowledge about having the disease given a positive tests result
where we shall not forget to take the prior, the prevalence expressed as
If we express the prevalence in odds,
we can update the prior odds of having the disease to posterior odds
where the Bayes factors are tidily separated. We can remember, that if we formulate the prevalence in odds, we can simply multiply the Bayes factor with the prevalence to update our beliefs on the odds of having the disease after receiving a positive test result,
Receiving a positive test in our example changes the prior odds from 10 to 990 (1 in 100) to the correct answer (C) 1 in 10,
with a Bayes factor of
If we tell the story about medical tests in the framework of probability theory, conveying that we are updating our prior knowledge by doing a test, would we have fewer misinterpretations about the implications of medical tests? Frankly speaking, assigning priors is already hard enough. We need to factor in the prevalence, symptoms, or contacts for contagious diseases, ... Why are we making the test interpretation another hurdle if we can learn more about probability theory and how to use it for inference problems? This will be the red thread for the rest of this thesis. We will be using this beautiful framework, and find use cases of Bayes factors for different challenges in molecular biology.
Now that we have familiarised ourselves with the first and most important Bayesian statistics tool in this thesis, the Bayes factor, we can explore what else we can use it for. We mentioned earlier, that we can also describe learning processes in science with the posterior distribution, simply by exchanging events and their probabilities for data we are seeing and hypotheses we want to test. Although we will return to the medical test paradox in the Discussion of the thesis, we can leave this exact use case of Bayes factors as a test statistic for medical tests behind us and take a look at how to use Bayes factors for other hypothesis tests. Let's return to our card game example for that.
We are watching a game of 'above or below'. As described earlier, the tactics (or lack of tactics) of a child who has never played a card game before are probably very different compared to an adult who has been confronted with similar games many times before in their life. We want to see whether the lack of tactics in this game makes a difference, or whether an adult could relax and close their eyes during that game, meaning it is all about luck anyway. We can do that using Bayes factors.
We want to find an equation for a Bayes factor that helps us evaluate after each game which hypothesis is more likely to be true. Both players, adult and kid, are shown the same cards and they give their predictions independently. We want to know whether there is a difference in their successes and how it changes over time, with more and more games the kid is exposed to. If we let our players compete in many games we might also be able to observe the learning progress of the kid (or the adult) over time. We are going to get a limited amount of data, as we are only recording a few games with our players, and so we employ Bayesian statistics here to help us make sense of our observations.
In every game the players will be shown a card in each round (
We can formulate two hypotheses. Hypothesis
The simplest model to describe the process we are observing is a binomial model, as we are observing a probabilistic event with two outcomes: winning or losing. Hence, we can describe the relationships between success rate
This description implies, for example, that if we knew the success rate
!
Figure 7: Imagine we are watching two players with different tactics represented by different success rates, 0.5 and 0.7. With the help of the binomial distribution we can compute the probability of successes in
Before the players have played the game we could guess how well they will perform, but, typically, their success rates
As we want to compare our two players, child and adult, to see whether their success differs, we are, technically speaking, asking whether the successes of our players,
arose from the same success rates q (which we don't know), or not. This implies that we are comparing two models, in Hypothesis
Before we collect our data we don't know about the performance of our players. Our prior information only consists of the fact that we are dealing with a probabilistic event with two outcomes which we can describe with a Binomial distribution. We don't know about the outcomes.
Let's look at some results of the games, Table 3.
Table 3: Our players have played 3 games (10 rounds each) of 'above or below'. We have counted how many times they have guessed the right answer. Both players were shown the same cards and they chose their answers independently.
We can infer an expectation value
following Laplace's rule of succession to infer probabilities for events given limited observations.
How can we learn more about
From the binomial distribution above we know that the number of successes
To calculate the posterior we need to find all the building blocks of the equation, in other words, we need to find expressions for all of them. We remember, the posterior is determined by two factors, the prior probability
The likelihood is how we get data that we gathered in an experiment into the equation: We collect data (
Secondly, we need an expression to feed in our prior knowledge, a prior distribution -- a distribution to capture our knowledge about the system before we collect data or before we include more data and update our knowledge. We can choose whichever distribution best describes our prior knowledge of
where the hyper-parameters
Can you spot the similarities? The prior gives us the option to start with no knowledge or bias (called flat prior), where all outcomes are equally likely by setting
Figure 8: Beta distributions change their shapes according to their hyper-parameters
Finally, the evidence
To visualise this integration, you can picture that the evidence is the area under the curve of
Figure 9: We can calculate the estimated number of successes for any probability of success
Note, that this is not limited to 2 dimensional spaces and can be extended to 3 or more if there are several variables you integrate over,
Now that we discussed all the building blocks of the posterior, let's put them together for our two hypotheses.
Using the binomial likelihood, we can infer the posterior distribution over the success rates of our players from the recorded games, starting from a prior for Hypothesis
and two priors for two parameters in Hypothesis 2,
which we will with the results from the games we recorded by multiplying with the likelihood. The likelihood of
For Hypothesis 2, the likelihood of the parameters,
If we now put together our equation for the posterior of Hypothesis
Filled with the building blocks for the posterior for Hypothesis
Note, how the evidence (denominator) simplifies to a Beta function (magic trick of choosing a conjugate prior), an identity we recognise from earlier and how the posterior as a whole can be expressed as a Beta distribution.
Analogously, for Hypothesis
and using the same magic conjugate prior simplification tricks we end up with a posterior probability function,
We have learned now how to update our knowledge about the success rates of our players. But how can we compare the hypotheses now? We want to know which hypothesis is more likely to be true, given our data. We can find the ratio of the two probabilities to evaluate,
where
where all except one pre-factor,
Figure 10: The process we have gone through in this derivation was finding (1) a likelihood function that serves as a model for the process we are watching, (2) a prior expression to include knowledge, and (3) a posterior to express our updated belief.
Congratulations, you have just made it through the first Bayes factor derivation of this thesis. Two more to go! If we calculate the Bayes factors for our recorded card games we get the following table, Figure 11. Usually, we transform our Bayes factor to
Table 4: Our two players, a kid and an adult have played 3 games with 10 rounds each. We counted the successes of them correctly guessing whether the card is 'above' or 'below'. Afterwards we inferred an expectation value
The Bayes factor is the ratio of evidences for the two hypotheses which we wish to compare. If there is is an equal probability for both hypotheses, we will get a log Bayes factor around 0, meaning we cannot conclude any explanation of the data is more likely. As soon as we can find more evidence for one of the two hypotheses in the data given our model, the
Figure 12: Positive
Inspecting Table 4 after you have learned that
Table 5: The importance of repetition and necessary insights become very clear when we compare the Bayes factors in Figure 11 and this Figure 13. We have gathered more data and therefore more evidence, resulting in more extreme Bayes factors. This table makes us believe that at first, kid and adult have very different success rates, and the differences decrease with the number of games they play. Interestingly, and we will encounter this in more detail in Chapter 6, it does not matter if they are playing these 300 rounds in 3 games or 5 games or 10 games, our overall conclusions will be the same. If we want to observe the learning process of the kid, however, we need to look at the process over time, of course, instead of all 300 rounds at once.
Bayes factors can be easier and harder to calculate, depending on how easy or hard it is to find the building blocks for the posterior distribution, and if simplifications happen like in our example here. The derivation we worked through here was, of course, chosen to demonstrate how simple it can be. In Chapters 3 and 6 we introduce Bayes factors in the analysis of RNA-Seq data for two different purposes. Relatively speaking, both of these examples are still simple and we could find analytical solutions for them, which is not given in many other problems out there.
Maybe you got here, because you want to use Bayes factors to search for differentially expressed genes in your RNA-Seq data.
| If so, | If not, |
|---|---|
| I am proud of you, my dear reader, you couldn't make me any happier right now. What you have been reading is the Introduction chapter to my thesis on Bayesian inference in the Analysis of RNA-Seq data; Searching for differentially expressed genes is one of my projects. If you want to learn some more, I highly recommend reading our method comparison work too. | Cool that you are here, for whatever reason that I would love to hear about! What you have been reading is the Introduction chapter to my thesis on Bayesian inference in the Analysis of sequencing data in molecular biology. The central question of my thesis is: How can we extract knowledge about life from data? We are nearly exclusively looking at a very popular data type in molecular biology, RNA-Sequencing data, a technique to identify and count all RNA present in a biological sample. If you want to know more about that, you can keep reading my thesis, where we go into the introduction to the technology next, followed by the content of the thesis. |
In any case,
I want to reiterate to the beginning of the thesis and find motivation to use Bayesian inference to analyse RNA-Seq data.
We have learned in this introduction about some simple principle mechanisms and the underlying philosophy of Bayesian statistics. We have touched on two big strengths in approaching statistical analysis like that so far. Firstly, we can incorporate prior knowledge into the analysis (which is not only a valuable thought for the analysis but for research and life in general). By combining prior knowledge with new insights gleaned from the data, Bayesian statistics offers a more holistic and informed approach to inference, especially in situations with limited or noisy data. Secondly, the uncertainty of our measurement is quantified in the form of probability distributions, allowing for a more nuanced representation of the unknown parameters. The goal is to capture the complexity of real-world uncertainty, which brings us to biology. The Bayesian framework's ability to provide probabilistic statements about parameters and predictions offers an intuitive and comprehensive understanding of uncertainty, which is exactly what we need.
Currently, we are collecting a lot of RNA-Seq data in biology, bringing a huge amount of very interesting insights about molecular and cellular processes to us. Nevertheless, it is still a measuring technique with limitations, that is measuring probabilistic events in complex living systems. The process of the data collection is highly complex, involving many steps of chemical and computational innovation we are only capable to use since a few years. This is paired with the challenge of identifying and quantifying thousands of macro-molecules that are part of dynamic processes in life. Analysing this data is unbelievably fascinating, and involves adventures from 18th-century mathematics to 21st-century innovation.
- Gerd Gigerenzer. Calculated Risks: How to Know When Numbers Deceive You. Simon and Schuster, 2002. 328 pp. isbn: 978-0-7432-5423-6. [Google Books](http:/ / books.google.com/ books?id=KJ7nrlJqcRYC).
- E. T. Jaynes. Probability Theory: The Logic of Science. Cambridge University Press, Apr. 10, 2003. 764 pp. isbn: 978-0-521-59271-0. [Google Books](http:/ / books.google.com/ books?id=tTN4HuUNXjgC).
- Sir Harold Jeffreys. The Theory of Probability. Third Edition, Third Edition. Oxford Classic Texts in the Physical Sciences. Oxford, New York: Oxford University Press, Aug. 6, 1998. 470 pp. isbn: 978-0-19-850368-2. [Google Books](https:/ / books.google.co.uk/ books/ about/ Theory_of_Probability.html?id=_PuRmAEACAAJ&redir_esc=y)
- Robert E. Kass and Adrian E. Raftery. “Bayes Factors”. In: Journal of the American Statistical Association 90.430 (June 1, 1995), pp. 773–795. doi: [10.1080/ 01621459.1995.10476572](https:/ / doi.org/ 10.1080/ 01621459.1995.10476572).
- Devinderjit Sivia and John Skilling. Data Analysis: A Bayesian Tutorial. Oxford University Press, June 2006. [Google Books](http:/ / books.google.com/ books?id=lYMSDAAAQBAJ).