The fact that FF4 is compared to FF6 is

worthy.
1) Sure FF4 (not DS) is pretty linear, but I'm not sure how you could hold that against it, since even the more popular games (CT) is pretty linear. Sure there are a lot of optional quests in modern games with looser storylines, but they are not required to complete a game... everyone nowadays are in the MMORPG syndrome... just find/complete quest and claim a prize. It's the same thing here. I'm not sure that's "remarkably" better in any way shape or form.
2) Character distinctness is important, but if everyone obsesses over having characters be the same-powerhouse as the next guy, then I'm not sure why they even have names. The fact that I can get lame Cyan to case the most basic magic that I commonly use, it makes me just as valuable than the next guy, even if Cyan's Magic Power is the worst in the game (although, I forget if Gogo is worse). There's absolutely no challenge, especially since the game WANTS you to learn as much magic as you can, especially with the faster learning espers. Sure FF4 was designed to be fixed in so many ways... stats, spell learning... pretty much the progression of stuff. So what? You have to LEARN the characters in FF4 to be able to maximize them. Espers almost makes you want to LEARN how you max their stat growths or makes you want to find out how much faster you can learn spells at crack speeds. Skills be damned as their importance becomes SIGNIFICANTLY less once you gain magic. Are most of Sabin's Blitzes overrated? Did you lol @ Cyan's Swordtech speeds? How often did you use Celes's Runic to save you from the Brachiosaur's Ultima, only to be raped by Meteor? Seriously, people don't realize that the skills on the characters are virtually obsolete (for the most part) because of magic. In FF4, you knew that Rydia sucked balls offensively (for most of the game), but her magic was awesome. Everyone knew Yang couldn't avoid a magical cold to save his life. Tellah didn't impress anyone, unless you knew how to abuse Osmose and Cure and Cure3. But hey, if every character is the same, what's in a name?
Edit: Addendum
In regards to FF5, that game was distinct, somewhat when it came to jobs... however the characters ended up being "the same" in the end. You end up having to build essentially 2 fighters and 2 spellcasters... or if you're crazy, an "omni-fighterwiz". The whole point of spending time in a job is to maximize the skills of a character, whether active or passive, so that they become virtually overpowered at the end of the game. It's most certainly different and amazing. It's still not very comparable to FF4 nor FF6.
3) Character development only gets progressively better during the SNES series of games (and perhaps Squaresoft in general). FF4's character development was weak, but it was "good enough" I guess. By the time you get to FF6, the storytelling has gotten better. I'm not sure why there should be a comparison... games get better over time. If we thought FF4 sucked ass so bad in the past, there may never have been a FF6 to play!
I'm sure I have more, but there's too much ignorance in that thread to kill yourself with.