New developments in scientific thought and hypotheses testing are changing the ways we think about truth and certainty.

Not only do scientific decisions rely heavily on tools and procedures of quantitative analysis, but also on social life and daily decisions. I want to start with a story from the judiciary system about the misuse of an important quantitative tool of the scientific community, Hypothesis Testing. It’s about the overturned Amanda Knox case in Italy. In 2007, she was accused of killing her roommate and sent to jail. The odds of DNA matching in the case of accidental death were reported as incredibly rare by a forensic data analyst, who examined data with regard to incidents of crime, concluding that the accident is not statistically an accident. The judges then made their decision based on this rareness, a single value, named p-value. Later, this was overturned (New York Times, March 27, 2013), because of a misinterpretation by the judges and lawyers, of the rare probability. What factors forced the judiciary system to make this decision based on a single value alone? Honestly, this is the story of the century and a story of Hypothesis Testing. This article introduces the logic behind their decision, draws attention to the misuse of data, and mentions alternative logics to this traditional approach on the matter (in March 2013, Italy’s Supreme Court ordered a retrial and found the two suspects guilty on January 30, 2014).

What is Hypothesis Testing?

We live in a data-driven world. Statistical inference is the process of drawing conclusions or decisions from data. The Hypothesis Testing procedure in the Neyman-Pearson paradigm is one of the procedures of this process, widely used in the scientific community for over a century. In this mindset, two complementary hypotheses are first defined: one is the conventional thesis (the null hypothesis) accepted to be true by default; the other is the alternative thesis (the alternative hypothesis) that needs evidence from data to falsify the null hypothesis. It results in a single value, called p-value, which is calculated from the data using theoretical model assumptions. A p-value is a measure – in probability sense, ranging from 0% to 100% – of how much evidence you have against the null hypothesis. The smaller the p-value, the more evidence you have against the null hypothesis. One may combine the p-value with the significance level to make decisions on a given hypothesis. This is also called significance testing. It has an accept-reject mindset (dichotomous decision or binary logic); in such a case, if the p-value is less than some threshold (usually .05) then the null hypothesis is rejected. Basically, it is a black or white decision, without considering the contrasts between them. This interpretation has been widely accepted in the twentieth century of scientific research, and many scientific journals routinely publish papers using this interpretation for the results of hypothesis tests, even though there are current tendencies not to use this. Let’s take a look at the history of this black-white logic.

History

The history of the process of hypothesis testing starts with Fisher (1890-1962), around the early twentieth century. Later, the contributions of Pearson (1857-1936) and Neyman (1894-1981), who were early fathers of statistics, about the interpretation of the hypothesis tests were integrated into present-day applications. Fisher’s approach focuses on inductive inference about a single hypothesis, whereas the Neyman-Pearson approach informs future behavior based on a test using two complementary hypotheses (Newman 2007).

These approaches are strongly influenced by Popper’s logic of falsification. Falsification can be defined as the act of disproving a proposition, hypothesis, or theory. This logic asserts that sufficiently improbable events can be considered impossible. A p-value suggests whether a null hypothesis is sufficiently improbable to be considered practically falsified in the sense of a logical refutation (Newman 2007). When we come back to the Amanda Knox case, the decision the judges made and justified is followed from this falsification logic in hypothesis testing. Another criticism with this mindset is: how fair it is to rely on a single value that yields a rare result? Are we going to generalize one lucky drawing of raffle tickets to believe all tickets are winners?

Logics and proofs

We use logical flaws and biases in our daily life. Finding the logical flaw in scientific papers is something most readers aren’t interested in. Instead, the results are believed as a fact. One of the widely used flaws/biases is in seeking or interpreting evidence in ways that are partial to existing beliefs or a hypothesis in hand (Mercie and Sperber, 2011). We tend to prove or show the accessibility or superiority of arguments by bringing evidence. Failure to prove that a treatment – say, a low fat diet – is effective is not the same as proving it is ineffective.

Let me give another example to clarify this: The New York Times editorial news reported on February 9, 2006, “Millions of Americans have tried to reduce the fat in their diets, and the food industry has obligingly served up low-fat products. Yet now comes strong evidence that the war against all fats was mostly in vain.” Actually, in the hypothesis testing mindset, the evidence collected from research should either reject the null hypothesis which is “diet is ineffective,” or fail to reject it. The mindset in testing is not about finding evidence to support the null statement. It is also not about proving the alternative hypothesis, “diet is effective.” Rather, it is basically looking for evidences to falsify the null statement. In the report, the “evidence” the reporter meant should be against the null hypothesis, not against the alternative hypothesis. Data is collected to falsify the null hypothesis, not to prove its truthiness, because the null is already accepted to be true unless convincing evidence is collected against it.

Let me give another example of this paradigm. As Newfoundland (2013) described, a person is innocent until proven guilty by bringing convincing evidence. The jury can’t say “he is innocent,” instead, the jury uses evidence that produces reasonable doubt to reject his innocence.

Developments in inferential paradigms

The accept-reject paradigm in inference represents a conventional wisdom. There are other, or currently developing, paradigms in inference, too. One of the dominating paradigms is the Bayesian-likelihood approach. After enjoying much wider acceptance in social and natural sciences, the Bayesian method suggests different views of hypothesis testing. The Bayesian approach is a method of data analysis in which subjectivity, conditionality, or past information is used to update the hypothesis as additional evidence is acquired. This approach in hypothesis testing offers a dynamic structure in probability calculation so hypotheses, and decisions, are not seen as a static truth; instead, they are updated with current data and the decision is stated in the sense of likelihood.

In our court room example, the Bayesian inference is applied to all evidence presented, with the past information being combined with the current evidence. The benefit of a Bayesian approach is that it gives all historical information so the decision is unbiased all along as the past data (prior knowledge) is used correctly. This paradigm changes the way statistics are calculated and how the result and inference are interpreted. Currently it is widely appreciated in quantitative data analysis, especially after convenient software exists for its implementation. However, the criticism to this approach, made by many, is found in its subjectivity. Objectivity and handling prior knowledge is a concern here so that different people, having different opinions, may arrive at different results.

Another developing paradigm: Fuzzy Logic

Fuzzy logic is another method in quantitative decision making. In contrast to binary logic (yes-no, or accept-reject), fuzzy logic can be thought of as gray logic, which allows a way to express in-between data values. It emerged in the development of the theory of fuzzy sets, by Lotfi Zadeh (1965). Fuzzy logic is mostly seen as a branch of artificial intelligence that deals with reasoning algorithms used to emulate human thinking and decision making. It handles the concept of partial truth using linguistic variables, where the truth value may range between completely true and completely false. The concept of partial truth would be subjective and would depend on the observer. For example, for the temperature of the weather, we want to determine when to say cold, warm, and hot. The meaning of each of these concepts can be represented by a certain membership (called a fuzzy set). The concept of each would be subjective. One might consider cold for all values up to 40 0F. In Figure 1, the meanings of the expressions cold, warm, and hot are represented by functions mapping a temperature scale to “truth values” ranging from 0 to 1. A point on that scale has three truth degrees, one for each of the three functions (expressions). The vertical line in the figure represents a particular temperature that the truth values (see three arrows) are measured. Since the red arrow points to zero, this temperature may be interpreted as “not hot.” The orange arrow (pointing at 0.2) may describe it as “slightly warm,” and the blue arrow (pointing at 0.8) “fairly cold” (Fuzzy Logic, 2013, para. 6). A rule could then be adopted, like for example, if “slightly warm” then stop the fan.

Picture was obtained from

Fuzzy logic uses truth degrees as a mathematical model of the vagueness phenomenon, and it summarizes data analysis in facts with truth degrees, and leaves the decision to the observer. Its advantage is its ability to deal with vague situations with respect to linguistic variables. While the significance testing for a hypothesis declares one accept or reject, fuzzy logic allows for degrees of acceptance or rejection. However, in fuzzy logic, the notion of truth doesn’t fall by the wayside, but it is expressed in degrees and offers possibilities for different situations. Regarding inference, fuzzy logic uses the mindset ‘everything is a matter of degree and open to interpretation,’ and this mindset is also adopted to machine learning, which is considered a more suitable mindset with human reasoning instead of binary logic. The constraints in fuzzy logic are found in its tools and interpretations when complex inputs are considered.

The developments of paradigms in statistical inference have similar fates as in the developments of mathematics, geometry, and physics. According to the Euclidean parallel postulate, in space, there exist no two parallel lines that intersect each other. However, after Non-Euclidean geometry was developed in the nineteenth century, a wider geometrical and mathematical reasoning stemmed from it; accordingly, the geometry of the physical universe and particles came to be understood better. The development of Riemannian geometry, offering that distance properties might vary, resulted in the synthesis of diverse results concerning the geometry of higher dimensional surfaces and the behavior of geodesics on them. It also made Einstein’s theory of general relativity justifiable. Likewise, as time passes, we witness wider paradigms or logics that abandon or correct former approaches in scientific decision making tools. This is a good reason why teachers and professors should update their current teachings as to be consistent with convincing trends, as well as to train students to be ready for wider paradigms in the future.

Conclusion

In today’s scientific community, the way decisions are made and fallacies proved, are changing as new paradigms emerge. In order to make better decisions or to validate claims, many aspects and methodologies should be considered. One way to avoid mistakes as much as we can would be to expect fallacy points to exist in human mental processing during decision making, and improving and seeking better alternatives with well-established wisdoms.

(Thanks Ugur Sahin for reviewing the preliminary copy of this article.)

References

• Fuzzy Logic. In Wikipedia. Retrieved August 1, 2013, from http://en.wikipedia.org/wiki/Fuzzy_logic
• Kass, Robert E. 2011. Statistical Inference: The Big Picture 1. Statistical Science, 2011, Vol. 26, No. 1, 1-9, DOI: 10.1214/10-STS337, Institute of Mathematical Statistics.
• Mercier, Hugo, Dan Sperber. 2011. “Why do humans reason? Arguments for an argumentative theory.” Behavioral and Brain Sciences. 34, 57-111.
• Newfoundland, Jason. 2013. “The Cell Phone-Brain Cancer Controversy.” The Fountain Magazine, Jan-Feb, Issue 91.
• Newman, Michael C. 2008. “‘What exactly are you inferring?’ A closer look at hypothesis testing.” Environmental Toxicology and Chemistry, Vol. 27, No. 5, pp. 1013-1019, 2008.

The lead article in this edition describes how a believer is expected to respond in difficult times. Storms are temporal and breezes are eternal in Gulen’s worldview, so thus believers “are neither shaken by the noise and clamor that is being intentionally made around” them, nor do they “panic in the face of the dust that envelopes us.” And for him, believers should trust in God, equip themselves with wisdom, and try to live up to their beliefs, God’s good pleasure, and distinct cultural values.

What would you do if terrorists installed a bomb somewhere in your city and they do not tell you how to defuse it? This is what is called “the ticking time-bomb scenario” and some people believe that under such circumstances torture is legitimate, if as a result the terrorist is forced to speak. Drawing on this scenario, Ayse Meva Nur provides an interesting discussion in this issue and she touches on human rights conventions and what Islam says about torture.

The intense rivalry between the magicians in the movie The Prestige was fatal. For Veli Keskin, the rivalry in the world stage between various power centers is as tragic as that which existed between the magicians; however, the global one has cost millions of lives. Keskin’s analogy has produced an eye-opening essay in which we find both the magicians on the stage in theatre and those on the political scene using similar techniques in shaping the perceptions of their audience.

Imagine you find a gift in your mail box one morning. It is a beautiful shirt perfectly suited to your taste. But who sent you this shirt? How did he or she know what your preferences were? Was the sender a man or a woman? With limited information it is not possible to discover the relationship between the sender and the receiver. But one can easily infer that the sender is someone who loves the receiver and wants to please the receiver. The former perspective, which is positivist, is not sufficient for explaining many things in our lives; but with qualitative inference we can reach some conclusions. Huseyin Coskun gives this and many other examples and explains how “qualitative inference allows one to address the non-physical or metaphysical aspects of the physical phenomena by freeing all of our cognitive faculties.”

We have a book written in golden letters lying in front of us. The decoration of the pages is attractive and impressive.. the small diamonds that are spread out over the pages are so shiny.. the cover is attractive.. the pages are velvety like a rose.. the colors are lively and bright.. the letters have been written by a talented calligrapher and are so harmonious; each one is a work of art.. While one may be interested in the book and its material another will find understanding what has been written and its content more compelling…

We recognize and perceive that which exists and evaluate that which is happening around us by using our natural senses, intelligence, and experience; as a result we are able to make inferences about the world through methods of deduction or induction. Positivism claims that the only true and valid knowledge is that which is based on actual sensual experience. According to materialism, the only thing that can be truly proven to exist is matter. In general, in analyzing matter and physical phenomena the deterministic approach is employed; all events are thought to be unavoidable consequences of the preceding sufficient causes. Sometimes the deterministic descriptions and formulations may end up in chaos. Chaos theory deals with the chaotic character of such formulations qualitatively. The quantum theory could also be interpreted as a deterministic tool that describes physical phenomena, more specifically, the behavior of matter and energy, within a range of probabilistic likelihood.

Some philosophers tend to include other factors in the cognitive faculties and the means that shape human perception and cognition: for example, emotions and intuition (e.g. Kant, Bergson). This apparently is a diversion from materialism and the deterministic point of view. The human being is not only an elegant combination of atoms; we also have a part that is not a function of matter or time: and this is the soul. If we restrict ourselves only to the natural senses and matter, as the positivist and materialist approaches do, then severe limitations on human perception are imposed, as demonstrated below. In general, it is not possible to perceive the truth completely without using all of one’s cognitive faculties. The question is then, how can a new program of reasoning be systematized to combine the senses and mind along with the emotions and intuition in a coherent manner? This novel program, which is called qualitative inference, is defined in this article using examples; however, first there is a brief introduction to a related topic, known as qualitative reasoning.

Qualitative reasoning is a relatively new approach in the estimation of possible solutions to real-world problems through non-quantitative reasoning. This approach provides a program to reason about the behavior of physical systems, such as the physiology of the human body, automobiles, etc., especially in the case of inexact or incomplete data, or when there is a lack of the precise quantitative information that is required by conventional methods. Qualitative models are more able to express states of incomplete knowledge about continuous mechanisms than traditional models which have been developed for the analysis of physical phenomena, as the former guarantees the ability to find all possible behaviors consistent with the knowledge in the model.

To get a flavor of what is meant by qualitative reasoning, consider this everyday scenario: You are confronted with a water-filled pan on a lit stove. You can easily predict that the pan will warm up, which will warm the water. At some point, the water will start to boil, and the pan might eventually become empty. To make these predictions, the exact values of the variables involved, such as the amount of water, the temperatures of the stove and the water, or the boiling temperature need not be known.

Observing pouring rain and a river’s steadily rising water level is sufficient to make a prudent person take measures against possible flooding-without knowing the exact water level, the rate of change, or the time the river might flood.

Qualitative inference is a program of qualitative reasoning that extends inference to the non-physical aspects of physical information, including reason, the relation between subjects and objects, etc. The universe itself is a great set of information, but this may not be exact or complete enough for metaphysical inductions. Needless to say, any such possible inductions should be consistent with known physical information. Consequently, in qualitative inference, the two key points we are referring to, namely the use of incomplete or inexact data and consistency with physical knowledge, are crucial.

While the deterministic approach could be considered to be the interpolation of existing physical knowledge and information in order to justify what is known or to discover what is unknown, qualitative inference can provide a tool for the extrapolation of existing physical knowledge and information. It can be used to reach what is unknown, physically unobservable, or even impossible to validate, regenerate, repeat, comprehend, or completely embrace by using standard scientific techniques.

Qualitative inference

With some examples we will explain how qualitative inference can be applied to everyday, real life experiences for inferences and inductions that are beyond our physical world.

A coincidence or a gift?

Let us take the example of a package sent to a person in the mail. Let us specify that it is a shirt. Let us further specify that the color of the shirt is the favorite color of the receiver, it fits him very well, the design is his preference, and the brand and the fabric of the shirt are those that he generally chooses. Based on this information, as it is incomplete, it would be difficult to answer questions about the possible relationship between the sender and receiver, the purpose of this shipment, etc. from a positivist or deterministic point of view. Through qualitative inference one could conclude, however, that the sender is someone who knows and is close to the receiver, who would like to please him likely because the sender likes him, and that the shirt has been mailed as a gift. This inference is obviously consistent with the known information. If we were to exclude the sense of love, enjoyment, pleasure, and closeness between two people it would not be possible to make these conclusions.

The apple

The apple is one of the most abundant types of fruit on the earth. “An apple a day keeps the doctor away,” goes an old saying. Doctors in modern times agree that the apple is important as a nutrient. An apple with its skin includes about 4 grams of dietary fibers. Potassium, calcium, phosphorus, magnesium, and selenium are minerals that can be found in an apple in different amounts. It also contains small amounts of iron, manganese, copper and zinc. The vitamins contained in an apple include vitamins A, C, E, and folic acid, which is an important nutrient during pregnancy. While eating an apple, one also strengthens the gingiva gums often without being aware that this is happening. The different colors of apples are very pleasant, as are their smell. How many artists have used the apple as their subject, due to its compelling beauty? It leaves a pleasant taste on the tongue and in the mouth. Many cuisines would be severely handicapped without the apple as an ingredient. The size is just right to grasp in the hand, and its flesh is succulent to bite. Different types of apples give the feeling of experiencing a new thing, yet remaining within the limits of our acquaintance. Apples are attractive and important sources of nutrients for animals as well. There is so much more to say about the apple and its importance to human and animal life, about how it is a very good source of nutrients that matches our needs, and addresses our tastes and pleasures.

With this information, the positivist perspective looks at the apple itself, its attributes, nutritional value, material properties, as well as the physical functions of the human being, and their need for the nutrients that are naturally included in the apple. This physical information, although valuable, is incomplete in understanding the perfect correlation between not only the need of the human being, but also our sense of taste and the apple, as we have briefly summarized above. Through qualitative inference, however, one could conclude that apple is a gift not only to the human being, but to most other living creatures, much like the shirt that was sent as a gift in the example above. This conclusion does not include any contradictions with the known, but incomplete, physical information. Although the sender of this gift is not known, one can infer that such a being exists and this being knows the human very well and is able to make and send this gift via the trees; this being possibly likes human beings and others who benefit from the apple, and likes to please them. However, without reference to the senses of love, enjoyment, and pleasure, it would not be possible to derive such conclusions.

Robot vs. mother

Now, we will consider another event, that in which a mother prepares food for her child. If she is feeding the baby with a formula, the steps she is likely to follow are sterilizing the bottle, boiling the water, refrigerating the water or waiting for the water to reach the appropriate temperature to mix the formula with the water, and stirring it carefully and thoroughly to make this a pleasant experience for her baby. From time to time, the order, attention of the mother, and duration of the steps may change without a clear difference in the underlying conditions. If one were to study this case scientifically, from a positivist perspective, then one would study the temperature of the water, the hygiene of the bottle, the actions of the mother, etc. for some possible inferences. But, these would be far from being a complete account of the situation and cannot reflect the entire picture or truth. The main point that a prudent person can derive from these scenes is clearly the mercy, compassion, and love of the mother towards her baby. This is the fact and main motivation that ensures that the mother take the actions mentioned above voluntarily, willingly, carefully, and repeatedly.

The Earth’s circulatory system

With this example in mind, let’s now review what we learned in high school about the earth’s circulatory system, a wonderful mechanism that purifies, cleans, and processes the water, the most important factor of the life. Human beings can survive for only a few days without water. Water is stored under the ground in natural reservoirs, whose formation alone is the topic for a series of scientific articles. Although the temperature increases as one approaches the center of the earth, sometimes the water comes to the surface at very cold temperatures; such cold clear water is the most desired substance on a hot, sunny day. Rain drops in their natural form are pure water and do not have a pleasant taste. These rain drops and the water used by people or animals is processed, that is, the water is cleaned, filtrated, and purified under ground. Minerals essential to human and animal life and some chemicals that naturally exist in the soil mix with the water. The addition of all these substances is good not only for the water’s nutritional value, but also creates a more pleasant taste. It is with these constituents that water becomes an essential nutrient for human beings and animals. Some of these additions are fluorine that strengthens the bones, fluoride, which is beneficial in low concentrations, sodium, potassium, and chloride, all of which play a role in the metabolism of the body. When the water comes to the surface as hot water, it can be used as a treatment for some diseases. The water is not processed only under the ground, as we have described here, but surface water is also atomized for the ultimate purification. When water evaporates, the vapor rises to a certain location in the atmosphere, accumulates there and forms clouds (which are a subject matter for poets, due to their beauty, especially when they appear on a sunny day). When water is condensed and catalyzed by the wind, the vapor returns to the earth as rain drops to complete the cycle.

Each step in the mechanism of circulatory system mentioned above can be analyzed scientifically, from a positivist perspective, to attain useful and important information. On the other hand, these results will never allow us to make inferences about the reason behind the perfect correlation between the needs of human beings and other living creatures and the availability of water in its most useful form as an essential nutrient with a pleasant taste, or the continuous availability of clean, fresh water for the living creatures who have no control over never-failing mechanism of the circulatory system, or the availability of water with the perfect chemical combination that allows coexistence of water in three different physical forms (ice, water, and vapor). On the other hand, using such incomplete physical information, namely the perfect match between the nature, properties, and availability of water, and the needs, the sense of taste of living creatures, a sensible person could easily infer that there is a hidden mercy and compassion acting behind the scenes, just as with the case of the mother preparing food for her baby. Without this qualitative inference, which is consistent with the known physical information and refers further to the sense of mercy and compassion, the entire picture of this phenomena and the truth can never be captured, regardless of scientific developments, the technology used in labs, or the computer systems that can carry out extremely complicated computations.

Conclusion

Qualitative inference leads us to arrive at conclusions about the non-physical or metaphysical aspects of physical phenomena. This program of reasoning considers physical phenomena and the universe as an incomplete source of information for understanding the metaphysical aspects of the physical world, helps us to make inferences that are consistent with this incomplete physical information, and to use all our cognitive faculties, including intelligence, senses, intuition, and even emotions, to arrive at non-physical inferences. Qualitative inference leads one to consider the apple as a gift and the availability of the fresh water through the natural water cycle as an indication of universal compassion and mercy for all living creatures; no living creature would be able to carry out such a process on their own. Without this reasoning, as exemplified above, only a very limited portion of natural phenomena can be understood sufficiently; the main points will always be missing or overlooked.

Through the same program of reasoning, a gift necessitates a loving sender, universal compassion and mercy requires a being that is compassionate and merciful. Just like the case studies presented above, all real life events have similar messages that can be understood by those who do not restrict themselves to physical nature. All physical events, for example, indicate a universal consciousness that implies the existence of someone who is conscious of everything. Such a being should be able to do all these things according to His will, so He is All Mighty… These examples shed light onto some of His attributes, such as His mercy, compassion, will, might, and consciousness. When other phenomena are analyzed in a similar manner, one can discover other essential attributes of this Supreme Being. As a result, qualitative reasoning not only proves the existence of the Supreme Being, it provides a useful tool in understanding His attributes.

Similarly, the rebirth of trees, insects, and the earth as a whole every spring after their death in the winter is a physical phenomena that we have all experienced. This information, through qualitative inference, can lead us to understand the Hereafter, that is, the resurrection of all human beings in another world. All the attributes of the Supreme Being mentioned above also support this inference. Mercy cannot be absolute and universal, for example, if the oppressed people and tyranny are treated the same; unfortunately, this is what seems to be happening in this world.

In brief, positivist and materialist approaches impose severe limitations on human perception. In reality, all the cognitive faculties of the human being play roles in our evaluation of our being, our position and role in this world, and our relation to others. The truth can only be obtained by a person through the systematic use of all such faculties. Qualitative inference suggests a program for doing so. While the positivist approach puts mankind into the position of being an observer of the things they can touch or see by invalidating all other sources of knowledge, qualitative inference allows one to address the non-physical or metaphysical aspects of the physical phenomena by freeing all of our cognitive faculties, and in particular allows to be a believer while observing the universe in a similar manner.