The problem is that the image is not fully transparent where you think it is.
The problem when taking an image and removing the background with transparency is that you often use the magic wand and it works for 99% but the rest you have to fix manually.
What I personally prefer in these cases is that you take your image (the png version with transparency) then in Photoshop you add another layer and put the first layer with the image on top. Then you change background color on the second layer (the one you just created) with a strange color in regards to the image, so in this case i would test with yellow.
Then if you have everything correct you should not see your image and since part of it is transparent you will see the yellow color where the image is transparent. Now you can zoom in and check the "strange parts" of the image and then you will find the spots that are not transparent. Now you can use the eraser or magic wand and polish the edges so that you only see yellow where it should be transparent. When you are done you could test and change the yellow color on the second layer to red and just double check. Then it will probably work as a charm. Good luck! If it does not work after that, please give us more details and also add the png image.