hyper-scraper

Task manager

Example use of TaskManager

TaskManager.execute(task)

Task configuration structure example

{
  "download":{
    "url": "http://example.com/"
  },
  "map": {
    "heading": "#header_counters strong:first-child"
  }
}

expect to return:

{ 
  timestamp: 1553293395534,
  input: { 
     datastamp: 'f812d50186b3b892abc3a852f2fe1c5d',
     size: undefined,
     code: 200,
     path: '/' 
  },
  extraction:
  {
     datastamp: '2b6fc1614e3893667d26497a329e7667',
     data: { heading: [25839] } 
  }
}

where extraction is the data extracted from HTTP response.