Community / The Reporter / Science News | Daily news articles, blogs and biweekly magazine covering all areas of science - News Headlines

We monitor the news headline from this website which we then use to train our sentiment analysis engine to detect for bad news about publicly traded companies

Domain: sciencenews.org , Associated categories: News

Harvested on CSV JSON HTML

2,573

news_headlines	article_urls	origin_pattern	origin_url	createdAt	updatedAt	pingedAt

Data source unique ID

n4835_d28f9b2e9e5842793a66f11a8c2a19fbeses

Privacy

Public

Last ran status

COMPLETED

Last ran

2020-01-23 06:00:00 UTC

Crawl Frequency

Six Hourly

Urls to Crawl

Use default URL in recipe

{
    "engine": "nokogiri",
    "client_version": "4.0.13",
    "origin_url": "https://www.sciencenews.org/",
    "columns": [
        {
            "col_name": "news_headlines",
            "dom_query": "ul li div.views-field.views-field-title span a"
        },
        {
            "col_name": "article_urls",
            "dom_query": "ul li div.views-field.views-field-title span a",
            "required_attribute": "href"
        }
    ],
    "cookies": [
        {
            "domain": "www.sciencenews.org",
            "expirationDate": 1526727703,
            "hostOnly": true,
            "httpOnly": false,
            "name": "__smVID",
            "path": "/article",
            "sameSite": "no_restriction",
            "secure": false,
            "session": false,
            "storeId": "0",
            "value": "ead7d68269ead1f2e4c4a4fdef42f6fe860bb90aeb46693033f7cb9bd6fbe2d6"
        },
        {
            "domain": "www.sciencenews.org",
            "hostOnly": true,
            "httpOnly": false,
            "name": "has_js",
            "path": "/",
            "sameSite": "no_restriction",
            "secure": false,
            "session": true,
            "storeId": "0",
            "value": "1"
        },
        {
            "domain": "www.sciencenews.org",
            "expirationDate": 1555671704,
            "hostOnly": true,
            "httpOnly": false,
            "name": "__smToken",
            "path": "/",
            "sameSite": "no_restriction",
            "secure": false,
            "session": false,
            "storeId": "0",
            "value": "TaI9mcO8UfTfHeYrjsSjt1gB"
        }
    ]
}