A question that Jean S inquired about before we were so rudely interrupted. The expanation in Steig et al was:
Principal component analysis of the weather station data produces results similar to those of the satellite data analysis, yielding three separable principal components. We therefore used the RegEM algorithm with a cut-off parameter k=3…. A disadvantage of excluding higher-order terms (k>3) is that this fails to fully capture the variance in the Antarctic Peninsula region. We accept this tradeoff because the Peninsula is already the best-observed region of the Antarctic.
I’ve sent an inquiry to one of the coauthors on what they mean by “separable principal components” and how they determined that there were three, as opposed to some other number. I would have thought that “significant” would be a more relevant term than “separable”, but we’ll see.
In the figure below, I show AWS trends for regpar =1,2,3,4,5,6,16, 32. It turns out that 3 was very fortuitous choice as this proved to yield the maximum AWS trend, something that will, I’m sure, astonish most CA readers. For regpar=1, the trend was negative. (I can picture a line of argument as to why 1 is a better choice than 3 or 4, which I’ll visit on another day.) As k increased the trend returned towards 0. Thus k, selected to be 3 no doubt from the purest of motives, yielded the maximum trend. I guess that was a small consolation for the bitter disappointment of failing to “fully capture the variance in the Antarctic Peninsula region” and it was definitely gracious of Steig and Mann to acquiesce in this selection under the circumstances.
ASW Trends under different regpar parameters (“RegEM Truncated PC”)
The graphic shows results for a method slightly varied from RegEM TTLS – let’s call it RegEM Truncated PC. I’ll explain the differences tomorrow. RegEM TTLS is a pig as regpar increases. RegEM TTLS yields rank k reconstructions; “RegEM Truncated PC” also yields rank k reconstructions, that were only about 1% different in benchmarks. For 1-6, RegEM TTLS has a similar pattern, but so far we haven’t run RegEM TTLS with higher regpar values as it will be VERY slow. (Jeff Id is going to try.)
I’ve got a bit of a personal interest as to why they excluded the PC4. Seems like something we’ve visited before, doesn’t it.
Feb 24 Update: Jeff Id reports:
I ran reg EM up to regpar=14, after that the methods it uses to set up TTLS don’t work and the regpar is truncated back down. Speed wasn’t too bad, each iteration in the loop takes about 3 seconds but the trend didn’t converge easily. After 100 iterations the high order system didn’t converge but was creeping closer. At regpar=8 it did converge after about 65 iterations.
