On 13 Aug. 1794 he was created a peer of the United Kingdom, with the title of Baron Mulgrave of Mulgrave, Yorkshire. On 30 Dec. he took part in the debate on the address in the upper house, and defended the recent acquisition of Corsica. Lord Grenville described Mulgrave's performance as ‘the most brilliant first appearance in that house that perhaps ever was remembered’ (Phipps, Memoirs of R. P. Ward, i. 28 n.) He was gazetted major-general on 3 Oct. 1794, lieutenant-general on 1 Jan. 1801, general on 25 Oct. 1809, and became governor of Scarborough Castle on 20 March 1796. In 1799 he was sent on an abortive mission to the Archduke Charles's headquarters at Zürich, to concert with him operations in Switzerland against the French (Life of first Lord Minto, iii. 77n.) He also visited the camp of Suwaroff in Italy and the court of Berlin. On 7 April 1801 he declined the offer of the command of the troops in Ireland, and his military career was brought to a close. He continued, however, to act as one of the chief military advisers of Pitt, and, although holding no ministerial office, was his chief spokesman in the House of Lords until Pitt's resignation in 1801. During the period of the Addington ministry (1801–4) Mulgrave, following the advice of Pitt, supported the treaty of Amiens in the House of Lords (Parl. Hist. xxxvi. 175–7, 701–2). In constant communication with Pitt while the latter was out of office, he pressed him to return to power (13 Nov. 1802). During 1803 he frequently criticised Addington's policy with much severity, and incensed the king against him. But when Pitt's second ministry was formed in June 1804, Mulgrave obtained the office of chancellor of the duchy of Lancaster, with a seat in the cabinet, and was sworn of the privy council. In the following January, when there was talk of Pitt's retirement, Mulgrave declared he would on no account serve in a ministry without him.
On 11 Jan. 1805 Mulgrave was raised to the responsible office of secretary for foreign affairs. The post was generally thought to be beyond his powers. T. Grenville, writing to the Marquis of Buckingham, expressed an opinion that he was only ‘put in ad interim until Lord Wellesley's arrival, who is expected in June’ (Courts and Cabinets of George III, iii. 404; Stanhope, Life of Pitt, iii. 161 n., 404). Mulgrave, however, showed himself fairly capable in debate. On 11 Feb. 1805 he had to announce the breach with Spain, and to defend the seizure of the treasure ships at Ferrol before the declaration of war (Parl. Debates, iii. 338–44), and on 20 June to defend the coalition of 1805 (ib. v. 465–7; Alison, Hist. of Europe, vi. 364–365). He composed an ode on the victory of Trafalgar (see Phipps, Memoirs of R. P. Ward, i. 171–2; Stanhope, Life of Pitt, iii. 371), and it was set to music by Dr. Arne. On 23 Jan. 1806 Pitt died. On 28 Jan. 1806 Mulgrave laid before the lords copies of the treaties recently concluded with Russia and Sweden, to which Prussia and Austria had acceded, and on 4 Feb. he explained their object. Three days later, on 7 Feb., he resigned, with the bulk of those who had been Pitt's friends.
While Lord Grenville's ministry of ‘All the Talents’ held office, Mulgrave took no prominent part in affairs. But on the formation of the Portland ministry in April 1807 he became first lord of the admiralty (cf. Parl. Debates, ix. 407–11, 590–1). His tenure of office was marked by the seizure of the Danish fleet, the Walcheren expedition, and the operations of Collingwood in the Mediterranean. He, Wellesley Pole [see Wellesley-Pole, William, Earl of Mornington], and an admiralty clerk, managed all the details of the Copenhagen expedition, and he sat up two or three nights copying out all the orders (Haydon, Autobiography, ed. Taylor, 2nd edit. i. 119). After the seizure of the Danish fleet Mulgrave offered a bounty with pay and victuals to three thousand Greenland fishermen to bring it to England. On 21 Jan. 1808 Mulgrave justified the expedition in the House of Lords (Parl. Debates, x. 31, 380–2, 656–8). On 26 Jan. 1809 he announced the determination of ministers to continue their support of Spain against Napoleon, and repudiated the theory that the British navy should be merely used as a home defence (ib. pp. 172–3). Mulgrave must be held to some extent responsible, owing to the obscurity and complexity of the admiralty instructions, for the comparative failure of the operations in 1809 against the French fleet in the Basque roads [see Cochrane, Thomas, tenth Earl of Dundonald; Gambier, James, Lord Gambier]. The misfortunes attending the Walcheren expedition he assigned to ‘adverse winds and unfavourable weather.’
Mulgrave retained his office under Portland's successor, Mr. Perceval, but resigned on the ground of ill-health in the spring of 1810. On 1 May he became master-general of the ordnance, still keeping his seat in the cabinet (Walpole, Perceval, ii. 79, 80; Phipps, Memoirs of R. P. Ward, i. 296). From this time he spoke rarely in the House of Lords. But after opposing the catholic demands in March 1812 (Parl. Debates, xxii. 60, 85), he in July supported Lord Wellesley's
