Characters? P4
Story? P5 (or maybe P3)
P4 is interesting to me as it blends comedy and serious rather well. There are times where that game can be really funny (the Kings Game is one of the best moments in any Persona game by far) but it does a good job balancing with the more serious moments.. After all, it's still a story about a serial murderer and a group of kids playing detective. There is a certain Scooby Doo appeal to that game in some ways.
Whereas P3 and P5 tend to be a little more straitlaced but maybe more interesting on a larger scope. P5 is an interesting reflection of living in modern day Japan and the issues prevalent to the country. Hence why I think the stories in both of those games are perhaps better than P4.
But P4 has so much damn charm and it really comes from that band of misfits. Like I said, I legitimately got sad come the end of that game as I REALLY didn't want to leave those characters....I've never felt that way about a video game before. Not even Shenmue or FF8 did that to me back in the day.
Although, P4 was the first one I ever played. And there is a good argument to be made that your first time will reflect your bias. Others swear completely by P3 or P5 as it was their first experience with Persona.
Anyways, I think all of them are totally worth playing in their own right...if you have the hours to spare that is and if the format is your cup of tea. They all have their own strengths and weaknesses.
I can't speak on P1 and P2...I've tried going back to those games on PSP but I think it's the structure of P3 onwards that appealed to ,me most about this series. P3 took on a more slice of life live by schedule format and I really like that format...they should go and remake P1 and P2 with the current day Persona format and have it all be cohesive as I would love to experience those games. But I just can't get into them. They feel a bit too dated for me.