Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ const metascraper = require('metascraper')([

- [metascraper-audio](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-audio) – Get audio property from HTML markup.
- [metascraper-author](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-author) – Get author property from HTML markup.
- [metascraper-citation](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-citation) – Get title, author, date, and publisher from Highwire Press `citation_*` and Dublin Core `dc.*` / `dcterms.*` tags.
- [metascraper-date](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-date) – Get date property from HTML markup.
- [metascraper-defuddle](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-defuddle) – A Defuddle connector for metascraper.
- [metascraper-description](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-description) – Get description property from HTML markup.
Expand Down
4 changes: 4 additions & 0 deletions packages/metascraper-citation/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Change Log

All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
59 changes: 59 additions & 0 deletions packages/metascraper-citation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<div align="center">
<br>
<img style="width: 500px; margin:3rem 0 1.5rem;" src="https://metascraper.js.org/static/logo-banner.png" alt="metascraper">
<br>
<br>
<p align="center"><strong>metascraper-citation</strong>: Get title, author, date, and publisher from Highwire Press <code>citation_*</code> and Dublin Core <code>dc.*</code> / <code>dcterms.*</code> meta tags.</p>
<p align="center">See our <a href="https://metascraper.js.org" target='_blank' rel='noopener noreferrer'>website</a> for more information.</p>
<br>
</div>

## Install

```bash
$ npm install metascraper-citation --save
Comment thread
coderabbitai[bot] marked this conversation as resolved.
```

Load it **before** the generic property packages so these values win over Open Graph (e.g. a clean `citation_title` instead of `og:title` with a site suffix).

```js
const metascraper = require('metascraper')([
require('metascraper-citation')(),
require('metascraper-author')(),
require('metascraper-date')(),
require('metascraper-publisher')(),
require('metascraper-title')()
])
```

## Supported tags

The bundle is a no-op unless [`test`](#test) finds at least one Highwire or Dublin Core tag. Highwire selectors run first; Dublin Core fills gaps (and covers repositories that only emit DC). Missing fields fall through to the generic rules.

Only the four properties below are set. Bibliographic extras (`citation_doi`, `dc.identifier`, ISSN, volume, issue, pages, PDF URL, language, abstract) are ignored.

| Property | Tags (first match wins) |
| --- | --- |
| **title** | `citation_title`, `dc.title`, `dcterms.title` |
| **author** | `citation_author`, `dc.creator`, `dcterms.creator`, `dc.contributor`, `dcterms.contributor` (first author only) |
| **date** | `citation_publication_date`, `citation_online_date`, `citation_date`, `dcterms.issued`, `dcterms.created`, `dc.date`, `dcterms.date` |
| **publisher** | `citation_publisher`, `citation_journal_title`, `dc.publisher`, `dcterms.publisher` |

Names are matched case-insensitively (`DC.Title` and `dc.title` are the same tag).

## API

### test

```js
const { test } = require('metascraper-citation')
```

Returns `true` when the Cheerio document has at least one `meta[name]` starting with `citation_`, `dc.`, or `dcterms.` (case-insensitive). Used as `rules.test` so the bundle skips pages with no scholarly markup.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## License

**metascraper-citation** © [Microlink](https://microlink.io), released under the [MIT](https://github.com/microlinkhq/metascraper/blob/master/LICENSE.md) License.<br>
Authored and maintained by [Microlink](https://microlink.io) with help from [contributors](https://github.com/microlinkhq/metascraper/contributors).

> [microlink.io](https://microlink.io) · GitHub [microlinkhq](https://github.com/microlinkhq) · X [@microlinkhq](https://x.com/microlinkhq)
50 changes: 50 additions & 0 deletions packages/metascraper-citation/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"name": "metascraper-citation",
"description": "Metascraper rules for Highwire Press citation_* and Dublin Core dc.* / dcterms.* meta tags.",
"homepage": "https://github.com/microlinkhq/metascraper/packages/metascraper-citation",
Comment thread
Kikobeats marked this conversation as resolved.
"version": "5.55.2",
"types": "src/index.d.ts",
"main": "src/index.js",
"author": {
"email": "hello@microlink.io",
"name": "microlink.io",
"url": "https://microlink.io"
},
"repository": {
"directory": "packages/metascraper-citation",
"type": "git",
"url": "git+https://github.com/microlinkhq/metascraper.git"
},
"bugs": {
"url": "https://github.com/microlinkhq/metascraper/issues"
},
"keywords": [
"citation",
"dublin-core",
"extract",
"google-scholar",
"highwire",
"html",
"metadata",
"metascraper",
"scraper",
"scholar"
],
"dependencies": {
"@metascraper/helpers": "workspace:*"
},
"devDependencies": {
"ava": "8",
"cheerio": "latest"
},
"engines": {
"node": ">= 22"
},
"files": [
"src"
],
"scripts": {
"test": "NODE_PATH=.. TZ=UTC ava --timeout 15s"
},
"license": "MIT"
}
11 changes: 11 additions & 0 deletions packages/metascraper-citation/src/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
declare function rules(): import('metascraper').Rules;

declare namespace rules {
/**
* `true` when the document has at least one Highwire (`citation_*`)
* or Dublin Core (`dc.*` / `dcterms.*`) meta tag.
*/
function test(htmlDom: import('cheerio').CheerioAPI): boolean;
}

export = rules;
64 changes: 64 additions & 0 deletions packages/metascraper-citation/src/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
'use strict'

const {
author,
date,
memoizeOne,
publisher,
title,
toRule
} = require('@metascraper/helpers')

const toAuthor = toRule(author)
const toDate = toRule(date)
const toPublisher = toRule(publisher)
const toTitle = toRule(title)

const meta = name => $ => $(`meta[name="${name}" i]`).attr('content')

const MARKUP =
'meta[name^="citation_" i], meta[name^="dc." i], meta[name^="dcterms." i]'

const test = memoizeOne(
$ => $(MARKUP).length > 0,
memoizeOne.EqualityFirstArgument
)

module.exports = () => {
const rules = {
title: [
toTitle(meta('citation_title')),
toTitle(meta('dc.title')),
toTitle(meta('dcterms.title'))
],
author: [
toAuthor(meta('citation_author')),
toAuthor(meta('dc.creator')),
toAuthor(meta('dcterms.creator')),
toAuthor(meta('dc.contributor')),
toAuthor(meta('dcterms.contributor'))
],
date: [
toDate(meta('citation_publication_date')),
toDate(meta('citation_online_date')),
toDate(meta('citation_date')),
toDate(meta('dcterms.issued')),
toDate(meta('dcterms.created')),
toDate(meta('dc.date')),
toDate(meta('dcterms.date'))
],
publisher: [
toPublisher(meta('citation_publisher')),
toPublisher(meta('citation_journal_title')),
toPublisher(meta('dc.publisher')),
toPublisher(meta('dcterms.publisher'))
]
}

rules.test = ({ htmlDom }) => test(htmlDom)
rules.pkgName = 'metascraper-citation'

return rules
}

module.exports.test = test
Loading
Loading