Monday, July 6, 2009

Microwave Water Experiment

On July 4th, I organized an experiment to compare the flavor of 4 different teas. Each tea was brewed with both microwaved water and electric kettle heated water. Six volunteers participated in the experiment. They were given two cups of each tea and then asked to rate the flavor of each tea on a scale of 1 to 10. They knew what tea they were drinking, but they had no idea which cup held the microwaved water and which held the kettle water. Without talking they wrote their ratings down on a post-it, then I collected the data and prepared the next round.

My lovely volunteers:
Their names (based on their Post-it colors) are:
Ms. Yellow, Mr. Yellow (no relation), Ms. Purple,
Ms. Pink, Mr. Orange and Mr. Blue

These variables were controlled:
1. Style of teapot (2-cup Chatsfords)
2. Same brewing baskets (brand new medium Finums)
3. Weight of dry leaf
4. Source of water (Crystal Springs)
5. Temperature
6. Heating material (Pyrex)
7. Steeping time

Heating the water in microwave:

Heating the water in kettle:

Brewing the tea:

I was not trying to prove anything by conducting this experiment. My goal with the experiment was simply to have a good time and to get some friends together to drink tea in an unusual format.

Most of us agree that natural elements are the key to brewing great tea, therefore, regardless of my results I wouldn't recommend using microwaved water to brew tea. That being said, I am also a big fan of the scientific method, and entirely curious (and clueless) about how this experiment would turn out. My hypothesis going in was: "very close results, slightly favoring the kettle."

The first tea was White Peony (百牡丹), a delicate white tea from Fujian, China.

Taster / Microwave / Kettle:
Ms. Yellow / 4.97 / 4.99
Mr. Yellow / 7 / 6
Ms. Purple / 5 / 6.25
Ms. Pink / 5 / 7
Mr. Orange / 4 / 6
Mr. Blue / 6 / 5
Total Rating / 31.97 / 35.24

The second tea was Joh Sencha, a vibrant, fresh tasting green tea from Japan.

Taster / Microwave / Kettle:
Ms. Yellow / 3.02 / 3
Mr. Yellow / 6.25 / 5.75
Ms. Purple / 7 / 7
Ms. Pink / 5 / 6
Mr. Orange / 7 / 4
Mr. Blue / 6 / 5
Total Rating / 34.27 / 30.75

The third tea was a delicious Spring 2009 Wenshan Baozhong from Pinglin, Taiwan.

Taster / Microwave / Kettle:
Ms. Yellow / 6.5 / 6.52
Mr. Yellow / 7.33 / 5.33
Ms. Purple / 7 / 8
Ms. Pink / 8 / 4
Mr. Orange / 5.5 / 9
Mr. Blue / 8 / 9
Total Rating / 42.33 / 41.85

The final tea was Keemun Hao Ya A, a rich black tea from Anhui, China.

Taster / Microwave / Kettle:
Mrs. Yellow / 6.5 / 5
Mr. Yellow / 1.99 / 1.98
Ms. Purple / 4 / 4
Ms. Pink / 1 / 2
Mr. Orange / 7 / 4.5
Mr. Blue / 7 / 8
Total Rating / 27.49 / 25.48

These results show very close numbers with microwaved water winning 3 out of 4 times and a combined rating of 136.06 for microwave and 133.32 for kettle. Because these ratings are so close and the sample was so small, we cannot make any real claims based on my experiment (except maybe that Ms. Pink hates Keemun)... but they do make you think!

In the end, we all had a fun afternoon and made some new tea friends. Most of the volunteers shared my hypothesis and, like me, seemed to be a little surprised by the results of this blind tasting.


Jason Witt said...

You're a real scientist. What I'd like to know is if you could control for how much those participants use microwaves at their own homes. If there's a lot of use there's going to be a bias there. I've never owned a microwave and have used an electric kettle for years so I'd probably prefer the kettle-heated water.

Nicole said...

I performed a student's t-test on your results and found a p-value of 0.89. The mean rating of the microwaved water was greater than kettle heated water(34.02 versus 33.33). This could indicate that in fact people lightly perfer microwave water. Might I suggest that next year you use the same tea over and over for many trials so that personal dislike of a certain tea won't influence rating. Exciting to see you doing some solid science! Love you!

Alex Zorach said...

Kudos on performing an actual statistical test. I haven't checked your math so I'll assume your figure is correct!

Keep in mind how to interpret p values: a p-value of 0.89 is universally interpreted as meaning that there's essentially no evidence that the effect is due to anything other than random variation. In general, you want the p-value to be small for the result to be strong. A typical "weak" p-value would be 0.05, which corresponds to data strong enough that truly random data would come out that way only about 1/20th of the time. Most hard science demands much lower p-values.

So I'd say there's no evidence that people prefer one water over the other. There's not even a weak trend, just random variability.