Ammonium nitrate requires multiple chemical processes before it becomes an effective explosive. As a fertilizer in farming, it is stored in large amounts in rural areas all around the world. That wouldn't be common practice if there was a general risk of it exploding.
The most likely scenario in Beirut seems to be that a large bomb was transported and assembled there. The fire that preceded the explosion was perhaps a decoy, along with some amount of fireworks thrown in as further decoy. And then the bomb was remote-detonated.
That's what a lot of the sources I've been reading have been saying. But it seems there are dozens of examples where ammonium nitrate on its own (or with another explosive material) caught fire and ended up exploding, including trucks that were transporting it, got in a wreck, caught fire, and then exploded after a while:
A truck carrying 12 tons of ammonium nitrate experienced an electrical fault and caught fire north of Taroom. After the driver stopped and parked the burning truck, two brothers from a nearby cattle property who saw the fire rode up on motorbikes to assist. The three men were killed when the truck exploded. The explosion burnt out more than 800 hectares (2,000 acres) of surrounding bushland, and left a deep crater where the truck had been parked. A memorial to the three men was unveiled at the accident site in 2013
The Roseburg Blast: A truck carrying dynamite and 4.5 tons of ammonium nitrate caught fire early in the morning of August 7, 1959. When it exploded it killed 14 people and injured 125 more. Several blocks of downtown Roseburg were destroyed. The accident is locally referred to as "The Blast".
The Norwegian cargo ship Ocean Liberty was loaded with 3,309 tonnes of ammonium nitrate and various flammable products when it caught fire at 12:30 July 28, 1947. The captain ordered the hold to be sealed and pressurised steam was pumped in. As this did not stop the fire, the vessel was towed out of the harbour at 14:00, and exploded at 17:00. The explosion caused 29 deaths and serious damage to the port of Brest.
Texas City disaster: The cargo ship Grandcamp was being loaded on April 16, 1947, when a fire was detected in the hold: at this point, 2,300 tons of ammonium nitrate in sacks were already aboard. The captain responded by closing the hold and pumping in pressurised steam. One hour later, the ship exploded, killing several hundred people and setting fire to another vessel, the High Flyer, which was moored 250 metres away and which contained 1,050 tonnes of sulfur and 960 tons of ammonium nitrate. The Grandcamp explosion also created a powerful earthshock that broke windows as far as 40 miles away and knocked two small planes flying at 1,500 feet (460 m) out of the sky. The High Flyer exploded the next day, after having burned for sixteen hours. 500 tonnes of ammonium nitrate on the quayside also burned, but without exploding, probably because it was less tightly packed. All but one member of the Texas City fire department died.
On April 4, 1925, and May 3, 1925, two carloads, each containing 220 barrels of ammonium nitrate, were dispatched from Muscle Shoals, Alabama, and caught fire in transportation. The barrels had been stored in a warehouse with varying humidity for 6 years, so it is believed that they were ignited by friction with their nitrate-impregnated manila paper lining. Other shipments were reportedly more successful.
If it was a bomb, perhaps the culprits knew about the nitrate and figured that into their plans. That would provide plausible deniability in the event that the nitrate didn't actually explode on its own.