This article is about How to do simple ruby web scraping by processing CSV.
In this article, we will create a Ruby on Rails application to scrap the link uploaded from a CSV file and find the occurrence of the link on a particular page.
In the below Ruby on Rails application development, the user needs to pass a CSV file and list of the user’s email to whom the parsed CSV will be sent.
In the CSV file, there will be 2 columns
Let’s start creating a Rails Application
$ rails new scrape_csv_data
$ cd scrape_csv_data
Then, we will generate an Upload CSV module. Run the below command.
$ rails g scaffold UploadCsv generated_csv:string csv_file:string
This will create all the required models, controllers, and migrations for csv_file. Run the migration using the below command.
$ rails db:migrate
gem 'carrierwave', '~> 2.0'
Then,
$ bundle install
$ rails generate uploader Avatar
class UploadCsv < ApplicationRecord
mount_uploader :csv_file, AvatarUploader
end
Update the routes.
Rails.application.routes.draw do
resources :upload_csvs
root 'upload_csvs#index'
end
Then, we will start the server and check if the application is working successfully or not.
$ rails s
$ rails generate job genrate_csv
gem 'httparty'
gem 'nokogiri'
After uploading the file check the scrap generated file will be updated. You can check the generated file in /scrape_data/public/result_data.csv
First, we will generate the mailer by using the below command.
$ rails generate mailer NotificationMailer
Add this code inside the notification mailer.
Also, we need to add mail configuration inside config/environments/development.rb or production.rb.
Also, we need to update the view also app/views/notification_mailer/send_csv.html.erb
app/views/notification_mailer/send_csv.html.erb

Thank you!
Source Link: https://www.botreetechnologies.com/blog/web-scraping-with-ruby-by-processing-csv/