Star Ocean 3 > Star Ocean 1 > Star Ocean 4 > Star Ocean 2
(SO5 isn't included because I don't have a PS4 and therefore haven't played it.)
Yes, you're seeing that right -- I ranked the fandom-darling SO2 lowest. That's not to say I dislike it, though! I think they're all great games! I just think SO2 gets put on too big a pedestal at times, so it feels to me like people forget that it has flaws of its own (main case in point to me: the way the plot revolves so much around Claude and Rena that the other playable characters feel like a bunch of irrelevant extras who are mostly there to fill up party slots). The fact that its stans can get pretty nasty about the other installments, especially SO3, doesn't help either.
Speaking of SO3, I think the fact that it was the first game in the series I played is a big part of why I rate it the highest; I guess it's the same kind of "first one I played wins" bias that I'm sure a lot of SO2 fans have. The game may have had its issues, especially in terms of the plot, but I got so utterly attached to it that I still feel a lot of nostalgia for it even now -- and it's been about 12 years since I was really into it. (It's scary to think that it's been that long!) That nostalgia's what inspired me to write this post out of the blue, actually.
As for SO1, something about the game and its characters just really endeared themselves to me, I guess; I wish it wasn't so ignored by the wider fandom. Because I enjoyed this game as much as I did, I loved getting to revisit Roak in SO4.
Truth be told, I still haven't actually finished SO2 or SO4, and because I've got other games on the go right now, I can't see myself doing so anytime soon either. Still, I'm nearly at the end of my first playthrough with Claude in SO2 and got pretty far in SO4, so I feel like I can make reasonable judgements as to how I rank them compared to the other games.