o.l.urlutils : module documentation

Part of otfbot.lib View Source

Some functions to simplify handling of HTTP-requests
Function get_page_with_header Download a web page as a string.
Function get_headers Undocumented
Function is_html Undocumented
Function convert_bytes copied from http://www.5dollarwhitebox.org/drupal/node/84
Function download_if_html download content, if the mimetype is text/html
Function download Uses twisted.web.client.getPage() to fetch a Page via HTTP
def get_page_with_header(url, *args, contextFactory=None, **kwargs): (source)
Download a web page as a string.

Download a page. Return a deferred, which will callback with a page (as a string) or errback with a description of the error.

See HTTPClientFactory to see what extra args can be passed.

def get_headers(url): (source)
Undocumented
def is_html(headers): (source)
Undocumented
def convert_bytes(bytes): (source)
copied from http://www.5dollarwhitebox.org/drupal/node/84
def download_if_html(url): (source)
download content, if the mimetype is text/html
def download(url, file=None, **kwargs): (source)
Uses twisted.web.client.getPage() to fetch a Page via HTTP
ReturnsA Defered which will call a Callback with the content as argument
API Documentation for OtfBot, generated by pydoctor at 2011-05-03 16:14:29.