How to only print a specific href from website

Question

I'm trying to make webscraping bots that join raffles automatically. I'm having an issue with only listing an href since there are 2 in the URL.

One of them leads to a page where I can join the raffle, while the other takes me to the profile page of the person who created the raffle.

Code:

URL = 'https://scrape.tf/raffles/ending'
pagee = requests.get(URL)
Soup = BeautifulSoup(pagee.content, 'html.parser')
for link in Soup.select('a'):
    print(link.get('href'))

Using the code above it prints the href for both the profile page and raffle page. What I want to do is make the code only print the href to the raffle page.

The href for profile page and raffle page are different:

profile page = /profile/(raffle creator's username)
raffle page = /raffles/######

I tried using the filter() function but it says:

TypeError: filtfunc() takes 0 positional arguments but 1 was given

When I use other codes in the filter() function it shows:

<filter object at ...........>

Is there a way to make Python only print the raffle href?

Please give a [mre]. It seems like filter is the right thing, and your last example shows you've successfully created a filter object, so what's the *problem*? — jonrsharpe, Jun 24 '20 at 07:12
Does this answer your question? [Calling filter returns ](https://stackoverflow.com/questions/12319025/calling-filter-returns-filter-object-at) — jonrsharpe, Jun 24 '20 at 07:16
Seems like this is what you are looking for: [Calling filter returns ](https://stackoverflow.com/questions/12319025/calling-filter-returns-filter-object-at) — castelon, Jun 24 '20 at 07:32

score 0 · Accepted Answer · answered Jun 24 '20 at 07:38

You can use re module and soup.find_all():

import re
import requests
from bs4 import BeautifulSoup


url = 'https://scrap.tf/raffles/ending'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for raffle_link in soup.find_all(href=re.compile(r'/raffles/[A-Z\d]{6}')):
    print('{:<40} {}'.format('https://scrap.tf' + raffle_link['href'], raffle_link.text))

Prints:

https://scrap.tf/raffles/V3CVJ3          I'm happy :D
https://scrap.tf/raffles/YE3QYW          Floofmancer's Hat Warehouse #17
https://scrap.tf/raffles/K71B2C          Daily smoothie #1
https://scrap.tf/raffles/FC331T          pan give pan 5#
https://scrap.tf/raffles/N7CQ19          FREE REF
https://scrap.tf/raffles/YWBTXH          it is
https://scrap.tf/raffles/1BGR3T          Day 2 of giving away items, while awaiting for the win in raffles
https://scrap.tf/raffles/X0VPFR          fire's Anxiety logs #151 - 152
https://scrap.tf/raffles/MGY62Y          Soldier's Daily Refined Giveaway
https://scrap.tf/raffles/VPAV8E          Soldier's Daily Reclaimed Giveaway
https://scrap.tf/raffles/8SJPL1          Soldier's Daily Scrap Giveaway
https://scrap.tf/raffles/P1NLD5          Random weapon №1
https://scrap.tf/raffles/Y7MVZU          Daily Refined #5
https://scrap.tf/raffles/89GXCH          What the hell do i do with this?
https://scrap.tf/raffles/I2DCRP          Greetings traveller, I have a quest for you.
https://scrap.tf/raffles/WAV5KI          Привет, товарищ.
https://scrap.tf/raffles/A8K38H          Playstation 3 messaging system scaled back - Article
https://scrap.tf/raffles/BZ9U1C          What is this very weird situation happening?
https://scrap.tf/raffles/3S0LVG          Pocket-Medes!
https://scrap.tf/raffles/PLDOFW           Rarity's Pride - Day 24 
https://scrap.tf/raffles/29S2I3          HAm 2
https://scrap.tf/raffles/A1P4C6          DR. MARIO?
https://scrap.tf/raffles/QWS50H          HAm
https://scrap.tf/raffles/EHT19E          Yo, take this. Do whatever you want with it.
https://scrap.tf/raffles/3FS1YC          This Gem, a result of my scout actions
https://scrap.tf/raffles/8JNRQG          Sandvich
https://scrap.tf/raffles/B45TAF          Gaming journalism was a mistake.
https://scrap.tf/raffles/0Q384E          Recommend me songs plsssss
https://scrap.tf/raffles/P6699P          Earthquakes are scary.
https://scrap.tf/raffles/AJZWMH          Day 1 of nothing to 10 ref
https://scrap.tf/raffles/BZB53B          Cheeto
https://scrap.tf/raffles/H5MUQB          Some Song Suggestions (MM's Hip Hop Related Raffles)
https://scrap.tf/raffles/DQTV49          Living Undead's life #12
https://scrap.tf/raffles/AXIUI6          I have not slept for the past 20 hours.
https://scrap.tf/raffles/FRSJT9          Is Min Min a Win Win? (Super Smash Bros Ultimate DLC 6)
https://scrap.tf/raffles/H517ZT          esta.
https://scrap.tf/raffles/5JVF21          Fish Fish Fish Fish Fish Fish
https://scrap.tf/raffles/TWKP0B          I want to make raffles
https://scrap.tf/raffles/DVJD2E          FUR sleepsies
https://scrap.tf/raffles/UFIRHI          This got me thinking...
https://scrap.tf/raffles/QT5ZGT          Me, actually making a daily raffle #141
https://scrap.tf/raffles/TBULX0          I won 3 raffles today, so I had to return the favor 
https://scrap.tf/raffles/CB5JD4           Neko Candy #536
https://scrap.tf/raffles/1P898Z          How many hobbies do you have bros? 
https://scrap.tf/raffles/UAMK5E          Bruh
https://scrap.tf/raffles/60PF0H          Using wish was a big mistake
https://scrap.tf/raffles/452V45          
https://scrap.tf/raffles/C8U1VL          Went to the hospital today and...
https://scrap.tf/raffles/IWY8OU          sandvich
https://scrap.tf/raffles/02D70K          When evening falls
https://scrap.tf/raffles/5KQ50T          Root n tooty point n shooty
https://scrap.tf/raffles/FY14DJ          By The Fireside #408
https://scrap.tf/raffles/NLRE8V          Spike's Scrap's #58
https://scrap.tf/raffles/PPD2YO          Spike's Musical Mayhem #25
https://scrap.tf/raffles/ZI4HHA          ◢◤ AVICII ◢◤
https://scrap.tf/raffles/GXOJ6U          Daily Nerdcore Mixtape #702) Macho Grubba (8-Bit Remix): Paper Mario TTYD - Tater-Tot Tunes
https://scrap.tf/raffles/ZBVWIO          Give the Darkest Dungeon Heroes a Pokemon Team: Jester's Finale
https://scrap.tf/raffles/1KBGV2          Hey
https://scrap.tf/raffles/QVIB72          Second raffle
https://scrap.tf/raffles/ANB2DK          I did

Just a quick question, if a button is not viewable in the screen (need to scroll to see), will python be able to interact with it? — KEnnnobe, Jun 25 '20 at 06:33
@KEnnnobe It depends. If the button is injected by JavaScript, then beautifulsoup won't see it. You have to use `selenium` or other means (`json`/`requests`/`re` modules etc.) — Andrej Kesely, Jun 25 '20 at 07:41
right now i have this. button = driver.find_element_by_id("raffle-enter"). then button.click but it says this error : selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable — KEnnnobe, Jun 25 '20 at 07:47
@KEnnnobe I don't have much experience with selenium, but I recommend to open a new question and describe there a problem (+ what have you tried so far). — Andrej Kesely, Jun 25 '20 at 07:48

How to only print a specific href from website

1 Answers1