Marceline's first words when Finn was in the lute suit outside her home weren't an immediate "No!" Instead she appeared shocked and told Finn "Trust me, you dont wanna date a girl like me." and proceeded to transform into a monster. It seemed more like she felt undeserving of love, even by someone with whom she feels close.
Just because a girl says she wants to be friends doesn't mean she isn't interested.