Fetching Data From Website Using Raspberry Pi

First we learned fetching data from any website using ESP8266-01 module along with Arduino Board. Then we grew to the 12th version of this board and learned to fetch data using ESP8266-12E board. Now let’s learn how to do this same thing using our Raspberry Pi board.

Let’s get Started

Fetching data uses one simple step and we can use that process on any board we want. That step is requesting an API. Yeah!!! request that link using your browser, esp board, Arduino or any other board and you are done, you can get your required data on the terminal window. So let’s see how to do this on our Pi board.(I am using Raspberry Pi 3 board)

First you need to get this “requests” module to be installed on to your board.

pip <span class="hljs-keyword">install</span> requests

After that write a simple program of GET request:

<span class="hljs-keyword">import</span> requests
r =requests.<span class="hljs-built_in">get</span>(<span class="hljs-string">'https://api.thingspeak.com/apps/thinghttp/send_request?api_key=J0EIT0XF42Y00RBJ'</span>)
<span class="hljs-built_in">print</span> r.<span class="hljs-built_in">text</span>

This is the API for current time in India. So when I run this python script I get this in response.

So successfully fetched data from the website. Now the question is, how to make our own API for anything we want?

Here is the solution.

Making an API

I already taught how to make our own API in my previous videos but still repeating in short about how to make an API for a particular data from a particular website. Just taking the above example, let’s make an API for current time in India.

For that first of all you need to go to www.thingspeak.com.

Sign In or Log In to your account. Then goto /Apps/ThingHTTP.

Then click on New ThingHTTP:

Then you will get a form like this.

Now you need to decide from which website you need to get the data. In my case, I’m fetching the current time in India from a website called, timeanddate.com.

After that right click on the required data to be fetched (in this case it is time) and click on Inspect.

Then right click on the data from the window on right side, and goto copy>copy Xpath:

Now in the form, you just need to enter 3 details, Name, URL of the page and Xpath of the data.

Save it and You will get a API link which you can use it for your RPi program to fetch the data. You can even request that link through your browser to test it.

So this is it. You have learned to fetch data through RPi and also how to make our own API. Hope you find it interesting.

Video Tutorial