Finding the Best Ticket Price - Simple Web Scraping with Python

One of my favorite parts of the summer is attending music festivals. Most festivals offer "early bird" tickets for a significantly lower price than general admission, however they typically sell out well before the actual event. Whether it is laziness, lack of money, or just plain stupidity I never seem to purchase these early bird tickets on time and have to look to different options. In recent years I have found success using Craigslist last minute, around a week before the festival, and getting tickets around or even lower than the early bird/pre sale prices. This year instead of sitting on craigslist day after day refreshing I decided to try and automate the process.

After looking at the structure of the Craigslist results page and messing around with BeautifulSoup I came up with the following script.

import requests
from bs4 import BeautifulSoup
from urlparse import urljoin

URL = 'http://philadelphia.craigslist.org/search/sss?sort=date&query=firefly%20tickets'
BASE = 'http://philadelphia.craigslist.org/cpg/'

response = requests.get(URL)

soup = BeautifulSoup(response.content)
for listing in soup.find_all('p',{'class':'row'}):
    if listing.find('span',{'class':'price'}) != None:
        price = listing.text[2:6]
        price = int(price)
        if price <=250 and price > 100:
        	print listing.text
    		link_end = listing.a['href']
    		url = urljoin(BASE, link_end)
    		print url
    		print "\n"

Requests is used to get all of the data from the webpage and then beautiful soup parses out everything I was interested in. Once the script is run it returns the most recently posted tickets between $100 and $250 with the price, listing title, location and link.

Using this script in conjunction with something like cron or osx's launchd you can have the script run a few times a day and the output emailed to you.

In the future I think it would be interesting to keep track of third party tickets sales as the event approaches on websites like ebay, stubhub, and craigslist and see when the best time to buy is. Something similar to this study on when to book a flight. Also a web app that allows you to search all three at the same time could prove interesting.

Any feedback is appreciated, you can reach me on Twitter or email me at danforsyth1@gmail.com.