About half of the medieval churches in Suffolk were dedicated to St Mary under a particular title or devotion. Churches not dedicated to Mary, would have contained a Marian shrine, generally at the east end of the south aisle. Some shrines became so popular that they were translated to buildings of their own. This may be how the shrine of Our Lady of Grace came to be. During the High Middle Ages, the shrine of Our Lady of Grace was second only to that of Our Lady of Walsingham.

The medieval town of Ipswich was a busy maritime centre of trade and shipbuilding. The inns and taverns of the town were full of pilgrims who flocked to the shrine of Our Lady of Grace in Lady Lane, near St Mary the Elm Church.

The shrine to Our Lady of Grace at Ipswich is first recorded in 1152.

The shrine was just outside the walls of Ipswich, Suffolk, England. The site of the original shrine was just outside the west gate of the medieval town wall of Ipswich, and is marked by a plaque and a statue of Our Lady on Lady Lane.

In 1297 the daughter of Edward I, Princess Elizabeth, married the Count of Holland in the shrine.

Among surviving mentions, it is recorded that Sir John Howard paid a visit in August 1463, when his main residence was at Stoke-by-Nayland. On that occasion he apparently left an offering of 2d. At a visit on 5 May, the now Lord Howard spent a total of 10s on a pilgrimage to the shrine. At a visit on 22 January 1472 he left 10d. John Howard was of royal descendant and at least by 1470 an extremely wealthy man, a grandson of Thomas de Mowbray, Duke of Norfolk from 1397, whose line had failed in 1476, and though the title had been held briefly by Prince Richard, one of the ill-fated little princes in the Tower, it was created again on 28 June 1483 for John Howard, who had shown himself a close friend and loyal supporter of the usurping King Richard III. It was, therefore, now as the new Duke of Norfolk that Howard visited the shrine again, on 16 August 1483, leaving an offering of 20d, then 4d "to bow on Owr Ladys fote" (presumably a particularl offering on kissing the foot of the statue) and also 11d "in almes at Owr Lady of Grace".[6] Howard was to die with his royal patron at the Battle of Bosworth on 22 August 1485.

After the Tudor dynasty had consolidated its hold on the English throne, Henry VII's Queen, Elizabeth of York, made a donation in 1502 of half a golden angel (3s 4d) to the shrine.

Between 1517 and 1522, Henry VIII and Queen Catherine of Aragon paid separate visits to the shrine, as did Sir Thomas More and Cardinal Thomas Wolsey, who was born in Ipswich.