How Science works - part 1

We are bombarded with information.

Traditionally this has been through print media, such as books and newspapers, and audiovisual through the medium of television.
However, in the last 25 years, the exponential growth of the internet has resulted not only in a huge increase in the amount of information, but it has also increased the number of sources of information.

Prior to the internet, most of our sources of information came from vetted sources, such as journalists and experts in their field. With the advent of the internet, and social media, anyone can now express their opinions and views, increasing the number of ‘voices we hear’.
Many of these lack  the necessary knowledge of expertise and qualifications.

The recent challenges of climate change,  COVID, and subsequently the vaccinations have shown how powerful many of those voices have become.

With many claiming to ‘have the truth’, how do we know what is the truth?
How do we separate fact from fiction? 
How do we assess the claims made?

There are three key principles  to consider when evaluating claims

• What is the evidence?
• Who is communicating?
• How is it communicated?

Over a series of three blogs, I’ll address these principles, and in the process hope to give insight on how science works.

The Evidence

No matter which side of the debate you are on, evidence reigns supreme, but what evidence, scientifically speaking?
 
Evidence is data driven scientific work, that is,
- data is collected, and lots of it,
- correlations are drawn between different variables, and
- models are developed that attempt to explain the connections.
As more data is collected those models are tested, modified if necessary, and at times, the models change if the evidence (read ‘data’) does not fit the model.
 
What is important  is how the evidence is collected, since it is imperative that
- the data is reliable
- the data is consistent with multiple measurement taken -  and valid
- the data is free from things that could skew the results, such as bias, incorrect methodologies, or other variables that are not taken into account.

Within science, there are very strict processes scientists use to ensure their data is reliable and valid.

---

An overview of the scientific method

1.  Scientists design their research (whether it by experiments, or data collection, for example) with the proviso that what they are looking for is not true. In essence, scientists try to disprove things, rather than ‘prove’ them right.
For example,  a scientist may want to investigate whether certain foods lead to increased heart disease. They will design their experimental work to show that  that there is no link! 
Why? 
Because of confirmation bias. 

If you set out to look for evidence that supports your idea (in Science that is called a hypothesis) , then you stand the risk of seeing trends in the data that support your idea, when it actually does not.

2. Any experiment and/or data collection must have LOTS of samples.

If a pattern exists then repeated measurements should lead to similar results and so by repeating the same experiment many times, scientists can increase confidence that the data is correct.
(this is what is meant by the term ‘reliability’)
Think of it this way.
You see a star rating for a film and it scores 8.0 out of 10.
Is it any good? Well that depends. If the number of ratings collected is only one, that is, one person gave it 8/10, that’s not enough data.
What about 10 people? 
What about if 1000 people?

The higher the number of samples taken, and if they are consistent,  the more reliable the data.
So when doctors say there is a strong link between smoking and lung disease, we take this seriously because it’s based on a huge amount of data collected, millions of people in close to 10,000 research papers.

3. When scientists design experiments, they often compare the data to a ‘standard.’ This is referred to as a control.
An example will help.
A scientist wishes to test a drug for a heart disease. They will examine  many  patients (to ensure reliability). 
They give the drug to one group and then to another group, they may give something that looks like the drug, but actually contains no active ingredients, (called a ‘placebo’)

When they get the results, they compare the two groups.

You will very rarely get a super clear delineation (such as, all the people who received the drug  improved, whereas all the placebo group did not). 
You will get some overlap; there will be some who received the drug but for whatever reason, did not improve. Similarly you might find that some in the placebo group improved.

But if a VAST majority do improve with the drug, then there is a strong link to the drug and improvement.

4. The next step is to communicate the results, and researchers  will then communicate their results by writing a paper and drawing conclusions. They will rarely speak in absolutes (“the drug definitely  works”). Rather, they will be cautious. They understand that, as with any research, more evidence is better. So they may conclude, for example,

”People on Drug A showed  a 25% improvement compared to a placebo”

5. Finally, the researchers will submit their work (or paper) for publication in a reputable journal.

Is that it?
 No. 

The journal will review the paper, drawing on the expertise of other scientists in the field to critically examine the paper. If that passes the evaluation, the paper is published.
Once published, other researchers can try to repeat the work, and will publish papers supporting the work, or critiquing the work, if they collect data that does not support the original paper.
This is what is  meant by a paper that is peer reviewed. It has gone through a very lengthy and comprehensive process to ensure that the data collected is done well and the conclusions drawn are appropriate and free from bias.

As you can see, the process is rigorous to ensure the evidence collected is reliable and valid.
So when you read online (or elsewhere) about some claim it is very important that any mentioning of evidence is backed up by quoting the actual research, that is,  peer reviewed papers , which means it is data driven.
It is not enough to say that evidence exists without actually quoting the actual research. In that case, it is just an opinion.

​In the next post we will examine the source of information, who is the author and whether they have the credentials for the information., so subscribe