PDO explains twentieth century warming?

Discuss life, the universe, and everything with other members of this site. Get to know your fellow polywell enthusiasts.

Moderators: tonybarry, MSimon

seedload
Posts: 1062
Joined: Fri Feb 08, 2008 8:16 pm

PDO explains twentieth century warming?

Post by seedload »

I read this online paper by Roy Spencer:

http://www.weatherquestions.com/Global- ... al-PDO.htm

Since the paper peaked my interest, I downloaded the data and played with it myself. Here is the PDO Index data I downloaded.

Image

Using ONLY these data, I was able to make a reconstruction of twentieth century warming completely based on the Pacific Decadal Oscillation (PDO). Below is a graph I created. The pink plot is the actual temperature record. The blue plot is my reconstruction.

Image

Pretty sick.

Don't get me wrong. I am no scientist. In fact, I am not very smart. I don't write well. I make many mistakes. Obviously, I am making a fundamental mistake in being able to show such a strong corellation.

I think my fundamental mistake (and Roy Spencer's) is in how I am treating the PDO Index. The PDO Index is, after all, derived from measurements of temperature. Now, the index pruports to remove the global warming signal by subtracting mean global temperature anomalies but I don't really understand the process. Probably, the warming is still in the data somehow and my method gets at it.

So, whoever wants to help me out or has any interest, please find the holes in what I did for me.

My reconstruction basically starts at a historical temperature and adds or subtracts a little bit to it each year depending on whether the PDO is in a warm or cool phase. I tweak the threshold of the PDO Index value I consider to be a warming and cooling and I tweak the amount of yearly warming and cooling each represent. From this I get a plot that is a good fit to the temperature record but lacks the spikes. Then I add the yearly PDO Index data back in (scaled) to show the decadal oscillations in temperature. I get a pretty crazy fit.

Remembering what I said about not being smart and not writing well, and understanding that I was just putting down a flow of conciousness rather than trying to create a good scientific paper, please take a look at my description of my entire process here and give me any feedback you want:

http://home.comcast.net/~charlesz/pdor.htm

The spreadsheet I used is referenced from this link as well.

Thanks,
seed

tomclarke
Posts: 1683
Joined: Sun Oct 05, 2008 4:52 pm
Location: London
Contact:

Re: PDO explains twentieth century warming?

Post by tomclarke »

Seed,

As requested, a critique.

You are taking a graph of data. Choosing 4 tunable parameters, and adjusting these to get best fit with another graph of data. Possibly your choice of a step-wise non-linear model (cool/ignore/heat] should also be considered chosen to fit.

The point is, it is easy to get data to fit other data if you have enough tunable parameters and the data is noisy so that exact fit is not required. Also your model is non-physical and in fact not even remotely plausible. Why should the variable PDO have only three distinct effects each point? Much more likely a linear effect.

Contrast this with the GC models. Yes they have tunable parameters - but they do not adjust these to make the results fit the temp curve. The tunable bits are all for separate physical processes which have independent data for tuning, and so tuning these parameters is not with hindsight of the wanted temp curve.

This is possible because the models represent physical processes.

best wishes, Tom


seedload wrote:I read this online paper by Roy Spencer:

http://www.weatherquestions.com/Global- ... al-PDO.htm

Since the paper peaked my interest, I downloaded the data and played with it myself. Here is the PDO Index data I downloaded.

Image

Using ONLY these data, I was able to make a reconstruction of twentieth century warming completely based on the Pacific Decadal Oscillation (PDO). Below is a graph I created. The pink plot is the actual temperature record. The blue plot is my reconstruction.

Image

Pretty sick.

Don't get me wrong. I am no scientist. In fact, I am not very smart. I don't write well. I make many mistakes. Obviously, I am making a fundamental mistake in being able to show such a strong corellation.

I think my fundamental mistake (and Roy Spencer's) is in how I am treating the PDO Index. The PDO Index is, after all, derived from measurements of temperature. Now, the index pruports to remove the global warming signal by subtracting mean global temperature anomalies but I don't really understand the process. Probably, the warming is still in the data somehow and my method gets at it.

So, whoever wants to help me out or has any interest, please find the holes in what I did for me.

My reconstruction basically starts at a historical temperature and adds or subtracts a little bit to it each year depending on whether the PDO is in a warm or cool phase. I tweak the threshold of the PDO Index value I consider to be a warming and cooling and I tweak the amount of yearly warming and cooling each represent. From this I get a plot that is a good fit to the temperature record but lacks the spikes. Then I add the yearly PDO Index data back in (scaled) to show the decadal oscillations in temperature. I get a pretty crazy fit.

Remembering what I said about not being smart and not writing well, and understanding that I was just putting down a flow of conciousness rather than trying to create a good scientific paper, please take a look at my description of my entire process here and give me any feedback you want:

http://home.comcast.net/~charlesz/pdor.htm

The spreadsheet I used is referenced from this link as well.

Thanks,
seed

bcglorf
Posts: 436
Joined: Mon Jul 23, 2007 2:58 pm

Details

Post by bcglorf »

The point is, it is easy to get data to fit other data if you have enough tunable parameters and the data is noisy so that exact fit is not required. Also your model is non-physical and in fact not even remotely plausible. Why should the variable PDO have only three distinct effects each point? Much more likely a linear effect.

Contrast this with the GC models. Yes they have tunable parameters - but they do not adjust these to make the results fit the temp curve. The tunable bits are all for separate physical processes which have independent data for tuning, and so tuning these parameters is not with hindsight of the wanted temp curve.


All true. The thing is though, that reconstruction of historical temperature records from proxies have tunable parameters and the data is noisy. When those reconstructions are the basis for verifying and tuning the GC Models I think it applies to them as well.

seedload
Posts: 1062
Joined: Fri Feb 08, 2008 8:16 pm

Re: PDO explains twentieth century warming?

Post by seedload »

tomclarke wrote:Seed,

As requested, a critique.
Thanks, Tom
tomclarke wrote: You are taking a graph of data. Choosing 4 tunable parameters, and adjusting these to get best fit with another graph of data. Possibly your choice of a step-wise non-linear model (cool/ignore/heat] should also be considered chosen to fit.
It is probably my communication skills that are lacking here. I did not do stepwise non-linear. I am multiplying the PDO Index by a scaling factor and adding it to temperature. This process is linear not step-wise.
tomclarke wrote: The point is, it is easy to get data to fit other data if you have enough tunable parameters and the data is noisy so that exact fit is not required. Also your model is non-physical and in fact not even remotely plausible. Why should the variable PDO have only three distinct effects each point? Much more likely a linear effect.
First, I disagree with the idea that the model is "non-physical" and not remotely plausible. The PDO is known to have warm phases and cool phases that change amounts of rain/clouds. Obviously, clouds are principle players in the reflection of visible light thus potentially affecting the energy budget. Do you believe that changes in cloud cover can not plausibly affect the earth's energy budget?

Second, I strongly disagree with the idea that you can use a few linear tunables to make noise into whatever shape you wish. In this case, only two parameters affect shape and they produce linear changes. I can't make the global warming signal out of noise with two linear parameters.

There are really only two tunable parameters that make much difference in the reconstruction.

Offset - offsets the PDO index to make a value of zero represent no accumulation of temperture. Values over zero cause accumulation ( amplitude considered) and values under zero cause accumulation of a reduction in temperature (amplitude considered).

The justification is that, if we assume the PDO can cause accumulation of temperatures, we don't know at what threshold accumulation starts. We tune to see what fits the data best.

Scale - I scale the PDO to adjust the amount of contribution to accumulated temperature phase shifts create.

The justification is that, well, of course you have to scale this by something. I am converting from an "index" to actual temperature. I have to scale. I scale to fit.

This one is interesting though. Because, since I have scaled to fit, future investigation could verify that the amount I scaled by fits the physical process I speculated on before. How much change in PDO Index cooresponds to how much change in reflected light. If I can get that number from another source and show that the number is consistent with the "tunable" I used - well, then I am on to something aren't I?


BTW, neither parameter really affects the shape of the reconstruction. The Offset just changes the orientation of the shape. The scale just changes the amplitude of the shape. It is kinda like playing with the controls on a CRT monitor.
tomclarke wrote: Contrast this with the GC models. Yes they have tunable parameters - but they do not adjust these to make the results fit the temp curve. The tunable bits are all for separate physical processes which have independent data for tuning, and so tuning these parameters is not with hindsight of the wanted temp curve.

This is possible because the models represent physical processes.
Agreed. So, if there is a physical process that can be shown to link PDO Index to changes in global temperature, and if the affect of that physical process can be shown to match the scaling I used, then we are on to something.

tomclarke
Posts: 1683
Joined: Sun Oct 05, 2008 4:52 pm
Location: London
Contact:

Post by tomclarke »

OK, if linear that is much better.

But it is still not the same as the way GC models have parameters tweaked.

The crucial issue if whether you can find control data to determine the tweaks that is unrealted to the final result (temperature). In other words because the model is Physical you can fit part of it to data on the underlying system (maybe relative humidity vs cloud cover) and then do the calculation to see how well the result fits temperature.

This is definition of physical model. Otherwise you have statistical model which suffers from potential over-fitting.

In your case, you would need to have non-temp data to tweak your intermediate result and then the final result (including scale factor) comes out of the physics.

At least this is my understanding and if GC models tweak based on required result of course they are not physical.

Best wishes, Tom

MSimon
Posts: 14334
Joined: Mon Jul 16, 2007 7:37 pm
Location: Rockford, Illinois
Contact:

Post by MSimon »

Seedload,

I like your revised explanation. Much clearer. Offset and gain.

Now the climate guys need to go to work and see if those are right. Shouldn't be too tough for those with skill in the field.

BTW I have been of the opinion for a while that if IPCC guys were using the PDO to explain the current cooling that they should go back and look at the warming too. Funny how they missed that.
Engineering is the art of making what you want from what you can get at a profit.

seedload
Posts: 1062
Joined: Fri Feb 08, 2008 8:16 pm

Post by seedload »

tomclarke wrote:The crucial issue if whether you can find control data to determine the tweaks that is unrealted to the final result (temperature). In other words because the model is Physical you can fit part of it to data on the underlying system (maybe relative humidity vs cloud cover) and then do the calculation to see how well the result fits temperature.

This is definition of physical model. Otherwise you have statistical model which suffers from potential over-fitting.
OK. Roy Spencer claims he can begin to do this with this small bit of data. Here is his diagram of PDO Index vs LS/SW fluxes measured by satellite.

Image

Not sure if I am buying this one yet. He says that satellite data is not good enough to measure flux accurately except in recent years, thus the lack of a good data trend.

He is showing changes in the energy budget unrelated to IR (CO2's friend) that track to PDO Index with an admittedly small amount of data and PDO tracking closely to global temperature.

MSimon
Posts: 14334
Joined: Mon Jul 16, 2007 7:37 pm
Location: Rockford, Illinois
Contact:

Post by MSimon »

Tom,

If the IPCC is using the PDO to explain the current trends they must have parameters. Make them cough them up.
Engineering is the art of making what you want from what you can get at a profit.

tomclarke
Posts: 1683
Joined: Sun Oct 05, 2008 4:52 pm
Location: London
Contact:

Post by tomclarke »

Simon -

If the models now incorporate the PDO I guess so. The detailed models are available surely? I will have to do some more wading through realclimate summary posts + 100s of comments looking at issue from all sides.

The trouble is that Climate Audit takes a few isolated issues and looks at them without bothering to understand the whole structure they are trying to shoot down. They give said issues great weight without looking at all the counter-evidence. Anyone can play this game in science.

To pick holes in the GC models & predictions it is necessary to be much more detailed and engage more fully with their content. I can't say I have done this so I am going on my judgement of others, based on their ability to engage in sustained open debate, rather than having direct evidence. I would not trust anyone who said they had first-hand understanding of the whole issue unless they had done a lot of careful reading (I know enough I think to detect flaws in this).

Not very helpful I know!

Seed -

I don't think this helps. instead of proposing a (statistical) model of correlation between PDO and temp he is proposing a (statistical)
model of correlation between PDO and LW/SW flux, which is strongly related to temp. Does he have physics equations which show why these things, not temp, relate to PDO, with a few unknowns? Otherwise he is just suggesting correlation between too simlilar looking graphs and the evidence as you say is very thin.

seedload
Posts: 1062
Joined: Fri Feb 08, 2008 8:16 pm

Post by seedload »

tomclarke wrote:Seed -

I don't think this helps. instead of proposing a (statistical) model of correlation between PDO and temp he is proposing a (statistical)
model of correlation between PDO and LW/SW flux, which is strongly related to temp. Does he have physics equations which show why these things, not temp, relate to PDO, with a few unknowns? Otherwise he is just suggesting correlation between too simlilar looking graphs and the evidence as you say is very thin.
Actually, Spencer does it differently than I do. Spencer uses an energy balance climate model and puts in forcing of cloudiness caused by PDO. He determines what the best fit PDO forcing is to produce the known temperature record. He then attempts to validate that forcing by looking at PDO to LW/SW fluxes. If they agree, then his forcing was legit.

He did it backwards.

If he had done his little study on PDO to LW/SW fluxes, derived a PDO forcing from it, and then plugged that forcing into a climate model he would match the methods you describe.

Backwards or forwards, it seems like the same thing to me.

What is bothering me isn't really procedural. It is related to the PDO Index itself. The PDO Index is derived from temperature. It is a spacial averaging of temperature attempting to compare warm to cold ocean waters. Negative when some waters are colder than average, positive when others are colder.

They pruportedly remove the gw signal by subtrating the mean global annomaly from those temperatures, but I can't get past the use of a temperature derived index being used as a correlation to... well... temperature.

If some of the gw signal is left in the data, will my method (or spencers) turn it into gain?

I am currently playing with some fake data. A sin wave is my PDO. I am adding a fake gw signal to it to create a temperature record to evaluate. I am taking away most but not all (95+%) of the gw signal to create my own little "PDO Index". I am then trying to get the gw signal back using my method on the PDO Index data. I can't do it. The signal is buried in the data. Without subtracting the sin wave again, I can't find the gw signal. I am still bothered but a little less so.

seedload
Posts: 1062
Joined: Fri Feb 08, 2008 8:16 pm

Post by seedload »

tomclarke wrote:If the models now incorporate the PDO I guess so. The detailed models are available surely?
You see, this is where I think models break down. In order for the models to 'incorporate' the PDO, there needs to be some physical process that causes the PDO that they can capture. But, as far as I know, the physical process that causes the PDO is unknown.

They can't model some cells getting warmer and some cells getting colder unless they have something that makes them do that. Right? So I doubt the PDO is even evident in the models. Then, if Spencer's theory has any weight, the models have to translate those warmer and cooler cells of ocean into more or less clouds in cells of the atmosphere and there needs to be a physical process captured to make that happen. Also, the clouds in the model now have to properly capture the reflectivity change based on these new clouds and their elevations.

Spencer is arguing that a 1% variation in reflectivity is enough to produce the warming seen in the last 100 years.

So, the models need to not only capture this physical process that creates the PDO, capture the physical process that creates more or less clouds at the proper elevations, and capture the physical properties of those clouds as far as reflectivity, but the model has to do so within 1%. The cumulative physics of the interactions have to be captured with a 1% tolerance!

I do not believe that this level of cummulative accuracy is possible.

MSimon, I start thinking about MTBF when I think through how all this potential for error adds up.

tomclarke
Posts: 1683
Joined: Sun Oct 05, 2008 4:52 pm
Location: London
Contact:

Post by tomclarke »

Seed,

I have to say I don't understand PDO yet. I have posted on another thread link to analysis of Spencer's data (maybe current, or maybe earlier than what he now says) which states his PDO index is a linear combination of two other indices.

Re uncertainty etc. It is only Spencer who things PDO has such big effect on temperature - again physically it seems unlikely it could have such a large effect. So although maybe there are big uncertainties in future PDO estimation this need not be a problem.

Don't forget that whatever the local movements of heat around the globe their effects tend to balance out, so that mean global temp is affected less than you might think, though obviously changes in cloud height etc can alter albedo and warming, so there could be effect.

MSimon
Posts: 14334
Joined: Mon Jul 16, 2007 7:37 pm
Location: Rockford, Illinois
Contact:

Post by MSimon »

It is only Spencer who things PDO has such big effect on temperature - again physically it seems unlikely it could have such a large effect.


Well no. The IPCC invokes it to explain the cooling trend which they expect to continue to 2015 (I think it will be longer).

In theory the IPCC is supposed to be the consensus. Of course if it is not then the whole shooting match is totally up for grabs. The consensus is breaking down. Probably a good title for a thread. I'll give it some consideration.
Engineering is the art of making what you want from what you can get at a profit.

MSimon
Posts: 14334
Joined: Mon Jul 16, 2007 7:37 pm
Location: Rockford, Illinois
Contact:

Post by MSimon »

Don't forget that whatever the local movements of heat around the globe their effects tend to balance out,
No they don't. The Southern hemisphere has been consistently cooler than the Northern for quite some time. I attribute it to cold air sinking. :-)
Engineering is the art of making what you want from what you can get at a profit.

tomclarke
Posts: 1683
Joined: Sun Oct 05, 2008 4:52 pm
Location: London
Contact:

Post by tomclarke »

What I mean is that regional variation is vey noticeable but less significant for the overall energy budget.

Here is an interesting perspective on politics & issues behind the latest IPCC report, Well worth reading, and links to the report itself.

http://scienceandpublicpolicy.org/repri ... eport.html

Best wishes, Tom

Post Reply