How to Quickly Ingest Data From a Feed Into a Database Without Coding

To get data ready for analysis as an SQL table, data engineers need to do a lot of routine tasks. This includes fetching the feed content via an HTTP client, parsing it into some structure, cleaning up and transforming the nested structure into a 2-D array, installing a database, and creating a table with a schema that matches the feed content or specific fields from it. Finally, the data needs to be inserted into the table. Only after that, the data is prepared and can be used in data analytics software. With TABLUM.IO, all of this can be done with just a few clicks.

TABLUM.IO allows data analysts to easily load data from remote websites and hosts via a URL. It supports data ingestion in JSON, XML, CSV/TSV formats, as well as through a REST API or SOAP request. The URL Downloader feature enables users to make multiple requests with data aggregation, custom HTTP headers, and parameterized loops for HTTP requests.

In this post, I’ll walk you through the simple process of importing data from an RSS feed.

The data source will be an RSS feed of Nutrition (Health/Living Well) from ScienceDaily:

The RSS feed is provided in RSS (XML) format and has a hierarchical structure, with elements and attributes arranged in a tree structure:

To start fetching data via URL into TABLUM.IO you need to click on the dropdown menu and select “URL Downloader”:

Paste the URL and click on the “Run Query” button:

Voila! We’ve got a fully featured SQL table out of a XML feed in just two clicks:

Let’s give our TABLUM.IO table a test drive. To get started, let’s click on the “Query via SQL” button to start querying:

In response, the system created a new TABLUM View with the SQL Console pre-filled with a default “SELECT” query for the entire SQL table. This allows us to start querying the recently loaded data from the RSS feed right away:

We’re dealing with an RSS feed about nutrition, and I’m looking to find all the studies that cover the topic of Vitamin D. I’m going to write the following query:

SELECT
  title,
  link,
  description,
  pubDate,
  guid
FROM
  DS_.V_1001
WHERE (title LIKE '%vitamin D%' OR description LIKE '%vitamin D%')
AND (title LIKE '%study%' OR description LIKE '%study%')

This query returned three records which contain the words “Vitamin D” and “study” in the titles or descriptions of the articles:

We may want to find out how many posts per month were included in the RSS feed. To do this, I’m going to run a query to see how many posts were published each month:

SELECT 
  toMonth(pubDate) AS month, 
  count(*) AS posts 
FROM DS_.V_1001 
GROUP BY month

TABLUM.IO returned my SQL query with a table that showed all the data I needed:

That’s all so far. Thanks for reading this post! If you’re looking to make your data preparation routine more productive and pleasant, come give TABLUM.IO a try — it’s free!

Next time I’ll be sharing more tips on how to make data preparation easier. Keep an eye out for our next post!