Yeah, Combat Power (CP) does matter. But the more important thing to win battles is choosing the right Pokémon. Each Pokémon has its strengths and weaknesses. If you choose the right Pokémon for battle, you can kick his ass even if the CP of your opponent's Pokémon is much more than your Pokémon.