The real reason of his absence, however, was that he had become convinced of the mis-government of the king, and decided as far as possible to withdraw himself from his councils (ib. p. 373). He did not come up to the parliament of 1254, but on this occasion he was unfit for the journey; for when, on the queen leaving England to join the king in Gascony at the end of May, he was again requested to take charge of the kingdom, he refused, feeling old age and sickness pressing heavily on him (ib. p. 447). However he attended the parliament which met on 6 April 1255, while he was at London. His anxiety about the affairs of the kingdom, conjoined with his habit of fasting, affected his head, and at the invitation of the Bishop of London he withdrew to Fulham for rest, and died there on 1 May, the third day after his arrival, having held the archbishopric for nearly forty years. His body was embalmed, conveyed to York with much honour by Walter, bishop of Durham, and buried in the south transept of the minster, under a monument with his effigy, which still exists. He published a body of 'constitutions,' probably in a provincial synod (Wilkins, i. 698).
In his diocesan work Grey was wise and active, and seems to have done much to reorganise the parochial system (Raine, p. 291). At York he built the south transept of the minster, probably founded the sub-deanery, and otherwise enlarged and enriched the prebendal body, and presented the church with a splendid set of copes and other ornaments. At Ripon he translated the body of St. Wilfrid to a new shrine (Metrical Chronicle, ll. 79, 385), and is said to have built the west front of the church. He also made some gifts to monasteries. He bought and attached to his church the village of St. Andrewthorpe, long known as Bishopthorpe, the residence of the archbishops, and a house in London, now Whitehall. This house was the residence of Hubert de Burgh, who gave it to the Black friars of London. Grey bought it from the Black friars, and it became the London house of the archbishops, and was called York Place down to Wolsey's time. He further provided a good amount of stock in all the manors of his see, and obtained an order from the crown that the same amount should be kept up by his successors. He died very rich, and left his private estates to his brother, Sir Richard Grey, with remainder to Richard's son Walter (Drake, Eboracum, p. 426).
Notwithstanding Grey's liberality to the churches of York and Ripon, he appears to have been harsh and illiberal in his dealings with the poor. This is proved by a story which, though it has some supernatural particulars, should not be discarded as 'ridiculously absurd' (Raine, p. 292n.), for it is told by Roger of Wendover (iv. 317) and accepted by Matthew Paris (iii. 299). Both take him as the most notable example of episcopal avarice, and relate that in a time of famine the stewards of some of his manors informed him that he had a quantity of wheat stored up which was perishing from age and vermin. Grey ordered that this damaged stuff was only to be given to the villeins on condition that they bound themselves after the next harvest to restore an equal amount of new grain. His steward at Ripon found the barn there full of toads and snakes. Nevertheless by Grey's orders his servants prepared to weigh it out to the poor; but it was found impossible to move it because of the stench, and a voice was heard saying: 'Put no hand on the grain, for the archbishop and all that he has are the devil's due;' so the grain was burnt to prevent the vermin from getting abroad. Moreover, Matthew Paris, in his notice of Grey's munificence at the marriage-feast of Alexander III, distinctly refers to reports as to his avarice (ib. v. 270). It is probable that the enormous sum which he had to pay at Rome for his promotion caused him to be over-strict in money matters during the earlier part of his archiepiscopate, and he may have changed in this respect in after years. He certainly changed in other ways, for that John liked and trusted him is sufficient to prove that he was at that time base and time-serving. In Henry's reign he helped to put English benefices into the hands of foreigners, and his refusal to accept an English clerk presented to a living (probably) Kirkleatham in Yorkshire by the patron, Robert Twenge, the famous 'Will Wither,' led to such serious consequences that the pope commanded him to accept the presentee (ib. iii. 217, 609-12). Towards the close of his life, however, he became dissatisfied at the evils of the administration, made no secret of his feelings, and was looked on as one of the most prominent of the patriotic party among the clergy. In this connection his name is honourably coupled with that of Bishop Robert Grosseteste, and men lamented his death as the loss of one who would not have shrunk from withstanding the oppressions of the Roman see. His position as a patriotic churchman gave rise to a story that he died under papal excommunication, and that consequently his body was not buried in consecrated ground, but laid within his monument above the level of the floor of the minster. Francis Drake [q. v.], the antiquary, made an opening in the stone work
