Sunday, August 23, 2015

Thresholdout - a stunning paper in Science

Fig 1 of Dwork et al. Their algorithm stops bogus
"learning" being "validated" from a holdout set -see text.
Stunning paper in Science called "The reusable holdout: Preserving validity in adaptive data analysis" by Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Aaron Roth.  This is about the serious problem that bedevils so much published research of over-fitting to data. As everyone knows if you try enough correlations between two datasets of any complexity you will eventually find one that has an apparently convincing p-value. This is sometimes called "p-hacking" and is a significant reason why so much published research is false.  And it is an even more serious problem for machine learning from "big data."

A sensible remedy for this is to divide the data into a training set and a "holdout set" which is only looked at when you have developed an apparently plausible hypothesis on the training set. But the trouble is - that only works once. If the holdout set doesn't validate the hypothesis and the researcher then tweaks the hypothesis the whole validity of the approach collapses.

Dwork and her colleagues have developed an algorithm they call Thresholdout which deals with this problem by injecting an element of Laplace Noise in the interrogation of the Holdout data set so that the over-fitting problem is avoided. Fig 1 shows what happens with a 10,000-point training and holdout set with a binary Y value that is in fact completely random. With the normal approach the "verified" accuracy of the classifier on the (repeatedly used) holdout set rises to over 60% (green line) whereas using Thresholdout it remains 50%.

I realised when writing this that Aaron Roth must be the son of Alvin Roth who I know slightly and like a lot. I finished his wonderful book Who Gets What and Why last month. On I said This is a simply brilliant book. Not only does it tell some fascinating stories which really make you think, and teach you a lot about the subtleties of how real markets - as opposed to the idealised "markets" of classical economics - actually work, but it is just beautifully written.

Friday, August 21, 2015

Wonderful Nielsen and Brahms Prom

 A lovely Prom last night, mainly Neilsen but some Brahms and some ... well we'll see.

The Danish National Symphony Orchestra was over under their excellent conductor Fabio Luisi who is music director of the Met. They began with a lovely Nielsen piece called Helios evoking the sunrise and eventually sunset over the Aegean. In the words of Psalm 19:

In the heavens he has set a tabernacle for the sun,
which comes out like a bridegroom from his chamber,
 and rejoices like a champion to run.
It rises from the far end of the heavens
and runs its circuit to the end again;
 and nothing’s hidden from the heat thereof.
Then we had an excellent performance of the Brahms violin concerto by Nikolaj Znaider, with a candeza by Heifetz (I remember years ago seeing this rather wonderful film with him playing himself). After rapturous applause Znaider came in for an encore and said "this will make some people angry, and other people very happy. The BBC have asked me not to play Bach, but I shall play Bach. He then played the Sarabande. Of course - anything else immediately after the Brahms  would be a come-down.

In the second half we had three unaccompanied motets by Nielsen and then an extraordinary and very moving work -  Hymnus amoris for Chorus, soloists and orchestra. This was inspired by Nielsen's wife, and the words were originally written in Danish but translated into Latin for the setting. It was inspired by this Titian which Nielsen and his wife saw on their honeymoon. Beginning with children (we were born from love, we grow up in love) and then mothers (you were born from love, you grow up in love) it finishes with old men and women. Really delightful and inspiring. The Sop Soloist was Anna Lucia Richter who "descends from a family of professional musicians" and I wonder if that includes the great Sviatoslav Richter (for whom Britten presumably wrote the cadenzas we heard the previous night) - I suspect it does, though the web is coy. Richter's mother was called Anna. I only heard him live once, in Kings College Chapel, and he played the Hammerklavier with, as an encore, the entire last movement again! 

Then Nielsen's Symphony No 2 The Four Temperaments which begins with tremendous gusto and was performed brilliantly. And finally as an encore A Dance of the Cockerels - from Masquerade. What an evening! I must listen to more Nielsen.

Thursday, August 20, 2015

Shostakovich 15 and Mozart K482 with Elisabeth Leonskaja

Super prom last night with the amazing Georgian Pianist Elisabeth Leonskaja playing Mozart Piano Concerto 22 in E flat major, K482, followed by Shostakovich 15 which I had never heard before.

There was an arrangement of Debussy's Petite suite by Henri Büsser which was pretty vapid I'm afraid.

The most amazing thing about the Mozart were the cadenzas, which are by Britten and presumably written for Leonskaja's colleague Sviatoslav Richter.  And she gave a terrific encore of a Chopin Nocturne.  I'd love to hear her play the Shostakovich Preludes and Fugues.

The Shostakovich 15 was also amazing though C says it came over very badly on the radio.

Here by request are some more photos of Leonskaja.

Thursday, August 13, 2015

John Scott RIP

The sad news arrived from a friend than John Scott died on Aug 12th. He was one of the world's greatest organists, who was a younger contemporary of ours at Cambridge, and who we got to know a bit when he was accompanist for the Bach Choir while C was a member.

I saw him a few years ago in NY rather unexpectedly where he had become Organist an Director of Music at St Thomas 5th Avenue. Apparently he returned to New York on August 11 after a very successful European tour. He was not feeling well the next morning and suffered a sudden cardiac episode. He was taken to Roosevelt Hospital but never regained consciousness. His wife, Lily, was by his side when he died. John and Lily are expecting their first child in September.

It is somehow fitting that tonight at the Proms they performed Messiaen's monumental Turangalia Symphony since Messaien was unquestionably the greatest organist/composer of the 20th Century (Bruckner of the 19th, JS Bach of the 17th, who of the 18th I wonder?)   I would certainly have gone had I not had meetings and heard this piece a few months earlier. I caught the end of it on Radio 4 and it was wonderful.

May he rest in peace and RISE IN GLORY.

And, if it works like that, may he already be enjoying conversations and music with Bach, Bruckner and Messiaen.

Tuesday, August 11, 2015

Psalm 37

An image of trust
Courtesy Wikipedia

Ah do not fret because of the unjust;
    and be not envious of wrongdoers,
2   away they soon will wither like the grass,
    and fade away from  view like the green herb.
But put your full trust in the LORD, do good;
    live in the land, enjoy security.
4    O make the LORD your joy and your delight,
    and he will give to you your heart’s desires.
Commit your way unto the LORD your God;
    O put your trust in him, and he will act.
6    He’ll make your vindication shine like light,
    the justice of your cause like noonday sun.
Dwell still before the LORD, wait patiently;
    don’t fret o’er those who prosper in their way,
    deviously achieving their success.
Eschew your anger, and leave rage aside.
    O do not fret—it only leads to harm.
9    For evildoers all shall be cut off,
    those waiting for the LORD shall gain the land.
10 For very soon the wicked will be gone;
    though you should seek them, they will not be there.
11 But lowly people shall possess the land,
    and in abundant peace they will delight.
12 Gnashing their teeth against the true in heart,
    the wicked plot against them spitefully;
13 But the LORD laughs at all the wicked folk,
    He sees their day of reckoning will come.
14 However much they draw the swords, and bend
   their bows to harm the poor, to kill the just;
15 their swords instead shall enter their own hearts,
    and their bows shall be broken utterly.
16 It’s better to have little, and be just,
    than the abundant riches of the wicked.
17 For weapons of the wicked shall be smashed,
    whereas the LORD upholds the righteous ones.
18 Just people have their lives in the Lord’s care,
    forever shall their heritage abide;
19  they are not put to shame in evil times,
    and in the days of famine they are full.
20 Killed are the wicked, enemies of the LORD
    like glory of the pastures they shall fade;
    like smoke they will all vanish clean away.
21 Lent to, the wicked folk will not repay,
    whereas the just keep giving generously;
22 for those blessed by the LORD shall have the land,
    but those accursed by him shall be cut off.
23  Made firm by our great LORD our steps are sure ,
    when in the way we choose he does delight;
24 and if we stumble, we shall not collapse,
    because the LORD supports us by the hand.
25 Now I am old, though I too have been young,
    yet never have I seen the just forsaken
    or seen their children begging for some bread.
26 The upright gives compassionately, and lends,
    their children are a blessing unto them.
27 O turn away from evil, and do good;
    and you will have a home for evermore.
28 Because  the LORD loves justice, and will not
  forsake or leave his faithful righteous ones.
Preserved for ever shall the righteous be,
    but children of the wicked are cut off.
29 The righteous ones shall inherit the land,
    and they shall live in it forever more.
30 Quiet wisdom shall be heard from righteous mouths,
    their tongues speak justice. 31 And the law of God
    is in their hearts; their footsteps do not slip.
32 Reprobate wicked keep watch for the just,
    and look out for a chance to murder them.
33 The LORD will not abandon them to their power,
    nor let them be condemned when brought to trial.
34 So wait upon the LORD, and keep his way,
    He’ll raise you up to inherit the land;
    and you will see the wicked ones destroyed.
35  The wicked I have seen oppressing, tall
     like lofty cedar tree from Lebanon.
36  Again I passed by, and they were no more;
    and though I sought them, they could not be found.
37 Use your eyes on the blameless, see the just,
     there is posterity for the peaceable.
38 But evil-doers shall all be destroyed;
    the offspring of the wicked be cut off.
39 Well-doers have salvation from the LORD;
    He is their refuge in the time of woe.
40 {Y} The LORD will help them and will rescue them;
    rescue them from the sinful, and will save,
    because they seek their refuge sure in him.

Monday, August 10, 2015

Rape of Lucretia at Glyndebourne

 Rather stunning live webcast of The Rape of Lucretia from Glyndebourne last night - which is available for a few days so you can catch up.

This was written for the great Kathleen Ferrier and first performed at Glyndebourne in 1946, so this is 69 years later. The cast and production is terrific.  Christine Rice plays the title role with Matthew Rose excellent as her caring and forgiving husband.

Fiona Shaw taking a curtain call
Tarquinius is portrayed convincingly by Duncan Rock and the Male and Female Chorus played by Allan Claytong and Kate Royal, provide the fascinating commentary and Christian framing of the work. Our friend Catherine Wyn-Rogers sang the faithful ancient maid Bianca who is also a mother-figure.

The librettist Robert Duncan is now almost completely forgotten - I had never heard of him - but on the evidence of this work he was very interesting.

Fiona Shaw directed brilliantly and took much deserved applause.

Catch it if you can!

Saturday, August 08, 2015

Wonderful Mahler 9 from the NYO

NYO of Great Britain
Listening on the radio to a phenomenal performance by the National Youth Orchestra of Great Britain of Mahler 9 at the Proms, conducted by Sir Mark Elder.  They have just executed the fiendish Rondo-Burleske with the most amazing panache, at a speed that makes no concessions to their age at all!

The contrast between the National Youth Orchestra of the USA, which managed the Emperor OK, but which couldn't in a million years have pulled of Mahler 9, is stark. The fundamental difference is that the UK NYO is a permanent ensemble which plays together all the year, whereas the US NYO is a scratch ensemble formed by the Carnegie Hall each summer. Given the vast size of the USA there is no practical alternative. But it just shows the value of practising together- as well as youthful enthusiasm.

I now bitterly regret not actually going to the Prom. Catch it on i-player.

I haven't got to Psalm 103 yet (only on Psalm 45) but it makes me think of
Making thee young and lusty as an eagle!

Tuesday, August 04, 2015

MacMillan and Mahler at the Proms

MacMillan and Runnicles take a bow
To our first Prom of the season to hear the world première of Sir James MacMillan's Symphony No 4 and Mahler 5, conducted by Donald Runnicles with the BBC Scottish Symphony Orchestra.

I really wanted to like and admire the MacMillan but in the end I'm sorry to say it missed the mark at least for me. There were some impressive moments and some very interesting ideas: the most compelling for me being the echos/quotations from Robert Carver and his 10-voice Mass Dum sacrum mysterium. But I'm afraid it all felt a bit like the Scottish Establishment talking to itself - it was a wonderful birthday gift to the wonderful conductor.

After the interval we had Mahler 5. Runnicles is a wonderful Mahler conductor - I will never forget the amazing Mahler 8 which closed the Edinburgh Festival - and of course Mahler 5 is a complete masterpiece. The tempi were a bit unusual but it was beautifully thought through, a very compelling interpretation and deserved its rapturous reception. Obviously the BBC SSO is not the LSO or the Vienna/Berlin Phil, so you don't quite get the utterly world-class strings or brass. But they played their socks off under their brilliant and beloved conductor, and it was a wonderful evening.

Remarkably, Mahler 5 wasn't performed in full in the UK until 1945, 43 years after it was composed, and wasn't given at the Proms until 1968 (under Boulez).

The concert was filmed and is available on iPlayer. Do catch it if you can. Both pieces are well worth listening to!  As indeed was the previous night's Prom (which we only caught on the radio) - Verdi Requiem with Runnicles and the BBCSSO. I didn't realise that Verdi wrote it as a tribute to Manzoni, the author of The Betrothed.

Sunday, August 02, 2015

A bad sermon but a great concert, reception and trip to China

HK at night
Back from an amazing 10 days in China - Beijing and HK. Most of this is unbloggable but I can tell you about:
  • The Eucharist in the Cathedral in HK on Sunday where the preacher, an English/Irish priest who was evidently trained in the 1980s. He was preaching on the John version of the Feeding of the 5,000 and was emphatic that "God does not feed the hungry."  He supposed "proof" was the Ethiopian famine which inspired Live Aid. He did admit that this is one of the very few incidents that is in all 4 Gospels, but he though that it was about Holy Communion and the idea of all people living together in peace and harmony.

    Now what's really frustrating is that even if you don't believe in miracles (and if you don't what on earth are you doing as an Anglican priest and why do you choose to preach about this passage?) you can still say many interesting things about this passage. You can say that the "miracle" was one of generosity, and you can explore the remarkable statement that Jesus realised that the people were going to take him by force and make him king, so he withdrew to a lonely place.  You could also explore the OT passage about Elijah feeding on a smaller scale (2 Kings 4:42-43). 
  • Yundi's concert in the HK cultural centre which was the culmination of his tour with the
    Tan Dun with conductor and NYOUSA
    National Youth Orchestra of the USA. He played the Emperor - magnificently, and it was so inspiring for the young players (who all wear red trousers and sneakers). Before that there was a piece by Tan Dun called Passacaglia: the secret life of birds which was remarkably moving and effective. In addition to orchestral playing all the kids had their mobiles pointed to a website which produced a twittering - and so did many members of the audience. It was very effective! Do it at the Proms!
  • Our reception for people (esp women) interested in being NEDs of UK companies. The Guest of Honour was an HK Billionaire philanthropist
    , the very impressive Consul General said a few words, and we also had the Nobel Laureate Sir James Mirrlees.
Back in the UK and had a wonderful but exhausting day sailing. And so to bed.