Loading IMDB data into SQLite in 200 lines of Python

You will be surprised how incredibly useful and powerful SQLite is.
data management

October 6, 2019

My wife and I love watching movies(mainly Telugu and Hindi) during weekends. I realized that we were wasting a lot of time reading reviews on the internet trying to decide which movies to watch. There definitely had to be a better way!

I knew that IMDB ratings were pretty reliable so I wondered if I could lay my hands on the IMDB ratings data. After doing some research on the internet, I stumbled upon this site IMDB datasets.

The data seemed fairly normalized so I wrote a quick and dirty script in python which downloads the zipped data files, unzips it and loads them into an SQLite database.

I am fairly happy with the result, it takes roughly 45 mins end to end from the point of downloading the files to loading the data into the database without any manual intervention.

Here are some highly rated Indian movies in 2019.

Original Title Genres
Joker Crime,Drama,Thriller
Ardaas Karaan Drama
Family of Thakurganj Action,Drama
The Sound Story Documentary,Drama
Shibu Comedy
Oththa Seruppu Size 7 Drama
Asuran Action,Drama
Digbhayam Thriller
Anandi Gopal Biography
Moothon Crime,Drama
Girgit Comedy
Adda Comedy
Avengers: Endgame Action,Adventure,Sci-Fi
Once Upon a Time… in Hollywood Comedy,Drama
Toy Story 4 Adventure,Animation,Comedy
Uri: The Surgical Strike Action,Drama,War
Gully Boy Drama,Music
Super 30 Biography,Drama
Article 15 Crime,Drama
Chhichhore Comedy,Drama
The Tashkent Files Drama,Mystery,Thriller
Super Deluxe Action,Crime,Drama
Nerkonda Paarvai Action,Drama,Thriller
Kumbalangi Nights Comedy,Drama,Romance
Jersey Drama,Sport
Yatra Biography,Drama
Agent Sai Srinivasa Athreya Action,Comedy,Crime
Sye Raa Narasimha Reddy Action,Drama,History
Unda Action,Comedy,Thriller
Virus Drama,Thriller
The Irishman Biography,Crime,Drama
Evaru Crime,Drama,Thriller
Portrait de la jeune fille en feu Drama,History,Romance
Thanneer Mathan Dhinangal Comedy,Romance
Uyare Drama
Section 375 Crime,Drama
Brochevarevarura Comedy
Seetharama Kalyana Action,Drama
Kavaludaari Thriller
Luca Drama,Mystery,Romance
The Gambinos Crime,Drama
Gang Leader Action,Comedy
Pengalila Drama
Rakshasudu Action,Crime,Thriller
Jiivi Drama,Thriller
Thamaasha Comedy,Drama
Chaal Jeevi Laiye Drama
To Let Family
Padmavyuhathile Abhimanyu Drama
Ilayaraja Drama
Chal Mera Putt Comedy,Drama
Ganagandharvan Comedy,Drama,Music
Jallikattu Crime
Mallesham Biography
Ninnu Thalachi Romance
Konttho Drama
Chasani Drama,Romance
Sikander 2 Drama
Vinara sodara veera kumara Drama,Romance
Penalty Sport
Network Drama,Thriller
Siege Sci-Fi,Thriller
Kempe Gowda 2 Action,Thriller
Arjun Suravaram Crime
Super Duper Comedy,Thriller
The Great Indian Escape Adventure,Biography,Drama
Has Jhan Pagli Fas Jabe Romance
Noblemen Drama
Nanna Prakara Crime,Mystery
Aamis Drama
Hansa Ek Sanyog Drama
Jacqueline I Am Coming Drama
Pratibha Drama
The Himalayan Diaries Comedy,Romance

Here is the script: gist

Back to top