I'm guessing it's to (1) allow backward compatibility, and because (2) balance is over-rated. Also, (3) katanas.(1) Earlier editions featured tridents. Lizard Kings used 'em (Hail Sakatha!), and there were magical tridents of submission and tridents of warning. Same with pole-arms. There were more types of pole-arms in 1e than playable character races. Heck, Gary Gygax wrote an essay on them (see appendices, 1e Unearthed Arcana). So, people converting earlier modules, or retro-jigging 5e to make it more like 1e, need some kind of a guide.
(2) Not every weapon needs to be mechanically balanced. They certainly weren't in earlier editions. Longsword was king in 1e. You'd have to be playing it purely for flava reasons to be picking a... say... pick [military pick]. Or horseman's mace. Or bec-de-corbin. I mean... there are reasons why medieval armies in our own world weren't marching to war with tridents. But tridents are flavorsome for your aquatic races. I don't care if a merfolk needs to spend +4gp for his trident! He's going to be using a trident because he'll just look kind of naff with a spear.
(3) Finally, and specifically referencing halberd vs glaive, I think it's the designers way of saying: "It's a long stick with a metal slicey bit on the end. Get over it." And I support that 100%. Personally, I really wished they had included a katana on the weapons list as well. 1d8 slashing damage, versatile (1d10), 3 lbs, 25gp. In other words, identical to a longsword, but +10 gp pricier. Because it's just a sword.
ncG1vNJzZmivp6x7prrWqKmlnF6kv6h706GpnpmUqHyouMCirZ5lpqh6qa3Lm5yrnF2staLA0maroZ1dmbanssSrnKeblWOAd4WVbm1o