v6.0.0

What's Changed

• Upgrade dependencies by @tamarasaurus in #44

Full Changelog: v5.0.0...v6.0.0

v5.0.0

What's Changed

• Bump css-what from 5.0.0 to 5.0.1 by @dependabot in #33
• Bump nth-check from 2.0.0 to 2.0.1 by @dependabot in #35
• Bump path-parse from 1.0.6 to 1.0.7 by @dependabot in #34
• Upgrade dependencies and switch to Jest by @tamarasaurus in #42
• Fix readme typo by @Calvein in #37

New Contributors

• @dependabot made their first contribution in #33
• @Calvein made their first contribution in #37

Full Changelog: v4.0.1...v5.0.0

v4.0.1

Puppeteer improvements:

• When initialising a scraper you can now pass launch options to puppeteer

const scraper = new Scraper('http://website.com', contract, { headless: false })

See the full list of options here.

• You can now ask puppeteer to wait for a selector to exist in the DOM before scraping the page

const scraper = new Scraper('http://website.com', {
  itemSelector: 'li',
  waitForPageLoadSelector: 'ul', 
  puppeteer: true
  attributes: {
    name: {
      type: 'text',
      selector: '.name'
    },
  }
})

• Bug fix: logic error in selecting the fetcher based on the puppeteer contract attribute has been fixed

v3.2.0

Update dependencies

v3.1.1

Expose a new function in Scraper getPageContents which returns:

{
  page: {
    contents: 'html contents',
    encoding: 'utf-8'
    url: 'http://page-url.com'
  },
  $: CheerioStatic object
}

v3.1.0

Add an option to use axios as a fetcher with puppeteer:false in scraping contracts

v3.0.2

Make schema more flexible

v3.0.1

Make selector in contract optional

v3.0.0

• Turn attributes from classes to functions
• Add schema validation for contracts
• Rename digit to number

v2.0.1

Updates

• Downgrade puppeteer to 1.14.0 until puppeteer/puppeteer#5674 is resolved