import requests
from bs4 import BeautifulSoup
from IPython.core.display import HTML
First, let's inspect the website we want to post to. This is a (nonfunctional) copy of our newsletter registration.
r = requests.get("https://www.physik.uzh.ch/~python/python/lecture_visual/test.html")
Since jupyter notebooks are rendered in the browser anyways, rendering a website is perfectly possible:
HTML(r.text)
In order find out where we need to post our content to, let's inspect the source code, as parsed by BeautifulSoup:
soup = BeautifulSoup(r.content, "lxml")
soup
You can use CSS selectors
soup.select("input")
Or normal methods like find_all
:
soup.find_all("input", attrs={"type": "text"})
But all we really need is the fact that the form gets posted to test.php
with three input fields called firstname
, surname
and mail
.
This we can pass as a dictionary in a call to requests.post
as the data
keyword argument:
r2 = requests.post("https://www.physik.uzh.ch/~python/python/lecture_visual/test.php",
data={"firstname": "Troll",
"surname": "McTrollface",
"mail": "spam@example.com"})
The result we get back is that we have successfully registered to the newsletter:
HTML(r2.text)