The are essentially the same, except for what you have to escape:
"'"
'"'
both work, but to incorporate multiple quote types you must escape the ones used to create the string:
"\"'"
'"\''
The two exist to make it easy for you to avoid having to escape your quotes, so the following two are easy:
'She said: "Not so fast!"'
"Won't you come with us?"
Note that there are also tripple-quote variants:
"""Now I can use either quote with more freedom: ' and "."""
'''Now I can use either quote with more freedom: ' and ".'''
These also allow newlines to be included without escaping:
"""A
multiline
string
is
easy.
"""
That last example would require you to use excessive \n
escape sequences otherwise.