I am following this tutorial and it’s just giving me errors on top of errors. I followed it verbatim.
When I run rake scrape I get back:
➜ scraper-2022 git:(main) ✗ rake scrape
HERE
rake aborted!
ActiveRecord::ConnectionNotEstablished: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:87:in `rescue in new_client'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:77:in `new_client'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in `postgresql_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `public_send'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:700:in `checkout_new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:679:in `try_to_checkout_new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:640:in `acquire_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:341:in `checkout'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_handler.rb:211:in `retrieve_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:313:in `retrieve_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:280:in `connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:568:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/attributes.rb:264:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/encryption/encryptable_record.rb:122:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:554:in `block in load_schema'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:551:in `synchronize'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:551:in `load_schema'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:417:in `attribute_types'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:443:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/type_caster/map.rb:16:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/arel/table.rb:107:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/table_metadata.rb:18:in `type'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:59:in `build'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:54:in `[]'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:126:in `block in expand_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `each'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `flat_map'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `expand_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:25:in `build_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:1299:in `build_where_clause'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:742:in `where!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:737:in `where'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/querying.rb:22:in `where'
/Users/DanielSkies/Desktop/scraper-2022/lib/tasks/scrape.rake:21:in `block (2 levels) in <main>'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:234:in `block in each'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:233:in `upto'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:233:in `each'
/Users/DanielSkies/Desktop/scraper-2022/lib/tasks/scrape.rake:14:in `block in <main>'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
Caused by:
PG::ConnectionBad: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/pg-1.3.5/lib/pg/connection.rb:702:in `connect_start'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/pg-1.3.5/lib/pg/connection.rb:702:in `new'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/pg-1.3.5/lib/pg.rb:69:in `connect'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:78:in `new_client'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:37:in `postgresql_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `public_send'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:656:in `new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:700:in `checkout_new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:679:in `try_to_checkout_new_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:640:in `acquire_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:341:in `checkout'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/connection_handler.rb:211:in `retrieve_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:313:in `retrieve_connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/connection_handling.rb:280:in `connection'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:568:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/attributes.rb:264:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/encryption/encryptable_record.rb:122:in `load_schema!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:554:in `block in load_schema'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:551:in `synchronize'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:551:in `load_schema'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:417:in `attribute_types'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/model_schema.rb:443:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/type_caster/map.rb:16:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/arel/table.rb:107:in `type_for_attribute'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/table_metadata.rb:18:in `type'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:59:in `build'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:54:in `[]'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:126:in `block in expand_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `each'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `flat_map'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:79:in `expand_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/predicate_builder.rb:25:in `build_from_hash'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:1299:in `build_where_clause'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:742:in `where!'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/relation/query_methods.rb:737:in `where'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/activerecord-7.0.3/lib/active_record/querying.rb:22:in `where'
/Users/DanielSkies/Desktop/scraper-2022/lib/tasks/scrape.rake:21:in `block (2 levels) in <main>'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:234:in `block in each'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:233:in `upto'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/nokogiri-1.13.6-x86_64-darwin/lib/nokogiri/xml/node_set.rb:233:in `each'
/Users/DanielSkies/Desktop/scraper-2022/lib/tasks/scrape.rake:14:in `block in <main>'
/Users/DanielSkies/.rvm/gems/ruby-3.0.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
Tasks: TOP => scrape
(See full trace by running task with --trace)
I did this many times, many different variations and still, it fails, What am I doing wrong?
Here is my scrape.rake file:
task scrape: :environment do
puts 'HERE'
require 'open-uri'
URL = 'https://jobs.lever.co/stackadapt'
doc = Nokogiri::HTML(URI.open(URL))
# doc = Nokogiri::HTML(URI.open(link))
postings = doc.search('div.posting')
postings.each do |p|
job_title = p.search('a.posting-title > h5').text
location = p.search('a.posting-title > div > span')[0].text
team = p.search('a.posting-title > div > span')[1].text
url = p.search('a.posting-title')[0]['href']
# skip persisting job if it already exists in db
if Job.where(job_title:job_title, location:location, team:team, url:url).count <= 0
Job.create(
job_title:job_title,
location:location,
team:team,
url:url)
puts 'Added: ' + (job_title ? job_title : '')
else
puts 'Skipped: ' + (job_title ? job_title : '')
end
end
end