The quarry was first worked on a small scale in the early 1840s. In 1864 William McConnel leased the quarry, forming the Aberdovey Slate Company Limited. McConnel planned to increase production at Bryn Eglwys. The limiting factor was the transportation of finished slates by pack horse to the wharves at Aberdovey. To overcome this, McConnel built the Talyllyn Railway, a narrow gauge line running from the quarry to Tywyn, where slate could be transhipped to the newly built Aberystwyth and Welsh Coast Railway.
The railway ran to , a lonely spot about 430 feet below the main quarry level. From there the half-mile long Galltymoelfre Tramway and a pair of inclines connected to the quarry.
Neither the quarry nor its associated railway were great commercial successes. By 1879 the company had run out of funds and both were auctioned off on 9 October 1879. After this and a subsequent auction failed to find a bidder, William McConnel personally bought both, and an upturn in the slate market allowed the quarry to expand further.
McConnel died in 1902 and the quarry became the property of his son W.H. McConnel. However the leases on the land the quarry occupied were close to running out and Bryn Eglwys ceased production in 1909.