How to Use Html5lib in Python

Written by mike wallace
  • Share
  • Tweet
  • Share
  • Email

The Python programming language can support HTML 5 web sites using the library 'Html5lib.' This library allows you to write Python scripts that parse HTML 5 pages using a tree structure. Tree structures are hierarchical views of web page elements. Accessing web page elements is accomplished using a tree walker. The tree walker 'walks' along the connections of tree nodes, and can traverse the entire tree. You can use Python with 'Html5lib' to open, view, and print out an HTML 5 web site.

Skill level:

Things you need

  • Python 3.2 programming language with Html5lib module

Show MoreHide


  1. 1

    Open the IDLE text editor in Program Files (or Applications for Macintosh) in the Python directory. A blank source code file opens.

  2. 2

    Import the "Html5lib" module by writing the following statements at the top of the source code file:

    import html5lib

    from html5lib import treebuilders, treewalkers, serializer

    import urllib2

  3. 3

    Create a new HTML 5 parser, which you will use to read an HTML website. Declaring a new parser by writing the following:

    parser = html5lib.HTMLParser()

  4. 4

    Open a website by passing its name into the urllib2.urlopen function. For example, if you want to open "," write the following:

    URL = urllib2.urlopen("").read()

  5. 5

    Pass the website into the HTML 5 parser to receive a tree representation. Save this representation into a variable named "tree" by writing the following statement:

    tree = parser.parse(URL)

  6. 6

    Create a tree walker like this:

    tree Walker = treewalkers.getTreeWalker("dom")

  7. 7

    Walk through the tree using the tree walker. The tree walker will return a stream of information that it discovers in the HTML 5 website. To walk through the tree, write the following:

    stream = tree Walker(tree)

  8. 8

    Serialise the stream so that you can easily output it to the console. You can serialise the stream using the following two statements:

    serial = serializer.htmlserializer.HTMLSerializer(omit_optional_tags=False)

    output = serial.serialize(stream)

  9. 9

    Iterate through the serialised output of the stream like this:

    for element in output:

  10. 10

    Indent the line immediately after the previous statement and write a print function, like this:


  11. 11

    Execute the program by pressing F5. The script will open and then parse an HTML 5 web page. The script then serialises the tree structure of the page and outputs it to the console. The output will vary depending on the web page chosen, but may look something like this:





    Welcome to a web page!



Don't Miss

  • All types
  • Articles
  • Slideshows
  • Videos
  • Most relevant
  • Most popular
  • Most recent

No articles available

No slideshows available

No videos available

By using the site, you consent to the use of cookies. For more information, please see our Cookie policy.