Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,7 @@ const metascraper = require('metascraper')([

- [metascraper-audio](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-audio) – Get audio property from HTML markup.
- [metascraper-author](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-author) – Get author property from HTML markup.
- [metascraper-citation](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-citation) – Get title, author, date, and publisher from Highwire Press `citation_*` and Dublin Core `dc.*` / `dcterms.*` tags.
- [metascraper-date](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-date) – Get date property from HTML markup.
- [metascraper-defuddle](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-defuddle) – A Defuddle connector for metascraper.
- [metascraper-description](https://github.com/microlinkhq/metascraper/tree/master/packages/metascraper-description) – Get description property from HTML markup.
Expand Down
4 changes: 4 additions & 0 deletions packages/metascraper-citation/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Change Log

All notable changes to this project will be documented in this file.
See [Conventional Commits](https://conventionalcommits.org) for commit guidelines.
59 changes: 59 additions & 0 deletions packages/metascraper-citation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<div align="center">
<br>
<img style="width: 500px; margin:3rem 0 1.5rem;" src="https://metascraper.js.org/static/logo-banner.png" alt="metascraper">
<br>
<br>
<p align="center"><strong>metascraper-citation</strong>: Get title, author, date, and publisher from Highwire Press <code>citation_*</code> and Dublin Core <code>dc.*</code> / <code>dcterms.*</code> meta tags.</p>
<p align="center">See our <a href="https://metascraper.js.org" target='_blank' rel='noopener noreferrer'>website</a> for more information.</p>
<br>
</div>

## Install

```bash
$ npm install metascraper-citation --save
Comment thread
coderabbitai[bot] marked this conversation as resolved.
```

Load it **before** the generic property packages so these values win over Open Graph (e.g. a clean `citation_title` instead of `og:title` with a site suffix).

```js
const metascraper = require('metascraper')([
require('metascraper-citation')(),
require('metascraper-author')(),
require('metascraper-date')(),
require('metascraper-publisher')(),
require('metascraper-title')()
])
```

## Supported tags

The bundle is a no-op unless [`test`](#test) finds at least one Highwire or Dublin Core tag. Highwire selectors run first; Dublin Core fills gaps (and covers repositories that only emit DC). Missing fields fall through to the generic rules.

Only the four properties below are set. Bibliographic extras (`citation_doi`, `dc.identifier`, ISSN, volume, issue, pages, PDF URL, language, abstract) are ignored.

| Property | Tags (first match wins) |
| --- | --- |
| **title** | `citation_title`, `dc.title`, `dcterms.title` |
| **author** | `citation_author`, `dc.creator`, `dcterms.creator`, `dc.contributor`, `dcterms.contributor` (first author only) |
| **date** | `citation_publication_date`, `citation_date`, `citation_online_date`, `dcterms.issued`, `dcterms.created`, `dc.date`, `dcterms.date` |
| **publisher** | `citation_publisher`, `citation_journal_title`, `dc.publisher`, `dcterms.publisher` |

Names are matched case-insensitively (`DC.Title` and `dc.title` are the same tag).

## API

### test

```js
const { test } = require('metascraper-citation')
```

Returns `true` when the Cheerio document has at least one `meta[name]` starting with `citation_`, `dc.`, or `dcterms.` (case-insensitive). Used as `rules.test` so the bundle skips pages with no scholarly markup.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## License

**metascraper-citation** © [Microlink](https://microlink.io), released under the [MIT](https://github.com/microlinkhq/metascraper/blob/master/LICENSE.md) License.<br>
Authored and maintained by [Microlink](https://microlink.io) with help from [contributors](https://github.com/microlinkhq/metascraper/contributors).

> [microlink.io](https://microlink.io) · GitHub [microlinkhq](https://github.com/microlinkhq) · X [@microlinkhq](https://x.com/microlinkhq)
50 changes: 50 additions & 0 deletions packages/metascraper-citation/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
{
"name": "metascraper-citation",
"description": "Metascraper rules for Highwire Press citation_* and Dublin Core dc.* / dcterms.* meta tags.",
"homepage": "https://github.com/microlinkhq/metascraper/packages/metascraper-citation",
Comment thread
Kikobeats marked this conversation as resolved.
"version": "5.55.2",
"types": "src/index.d.ts",
"main": "src/index.js",
"author": {
"email": "hello@microlink.io",
"name": "microlink.io",
"url": "https://microlink.io"
},
"repository": {
"directory": "packages/metascraper-citation",
"type": "git",
"url": "git+https://github.com/microlinkhq/metascraper.git"
},
"bugs": {
"url": "https://github.com/microlinkhq/metascraper/issues"
},
"keywords": [
"citation",
"dublin-core",
"extract",
"google-scholar",
"highwire",
"html",
"metadata",
"metascraper",
"scraper",
"scholar"
],
"dependencies": {
"@metascraper/helpers": "workspace:*"
},
"devDependencies": {
"ava": "8",
"cheerio": "latest"
},
"engines": {
"node": ">= 22"
},
"files": [
"src"
],
"scripts": {
"test": "NODE_PATH=.. TZ=UTC ava --timeout 15s"
},
"license": "MIT"
}
11 changes: 11 additions & 0 deletions packages/metascraper-citation/src/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
declare function rules(): import('metascraper').Rules;

declare namespace rules {
/**
* `true` when the document has at least one Highwire (`citation_*`)
* or Dublin Core (`dc.*` / `dcterms.*`) meta tag.
*/
function test(htmlDom: import('cheerio').CheerioAPI): boolean;
}

export = rules;
64 changes: 64 additions & 0 deletions packages/metascraper-citation/src/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
'use strict'

const {
author,
date,
memoizeOne,
publisher,
title,
toRule
} = require('@metascraper/helpers')

const toAuthor = toRule(author)
const toDate = toRule(date)
const toPublisher = toRule(publisher)
const toTitle = toRule(title)

const meta = name => $ => $(`meta[name="${name}" i]`).attr('content')

const MARKUP =
'meta[name^="citation_" i], meta[name^="dc." i], meta[name^="dcterms." i]'

const test = memoizeOne(
$ => $(MARKUP).length > 0,
memoizeOne.EqualityFirstArgument
)

module.exports = () => {
const rules = {
title: [
toTitle(meta('citation_title')),
toTitle(meta('dc.title')),
toTitle(meta('dcterms.title'))
],
author: [
toAuthor(meta('citation_author')),
toAuthor(meta('dc.creator')),
toAuthor(meta('dcterms.creator')),
toAuthor(meta('dc.contributor')),
toAuthor(meta('dcterms.contributor'))
],
date: [
toDate(meta('citation_publication_date')),
toDate(meta('citation_date')),
toDate(meta('citation_online_date')),
toDate(meta('dcterms.issued')),
toDate(meta('dcterms.created')),
toDate(meta('dc.date')),
toDate(meta('dcterms.date'))
],
publisher: [
toPublisher(meta('citation_publisher')),
toPublisher(meta('citation_journal_title')),
toPublisher(meta('dc.publisher')),
toPublisher(meta('dcterms.publisher'))
]
}

rules.test = ({ htmlDom }) => test(htmlDom)
rules.pkgName = 'metascraper-citation'

return rules
}

module.exports.test = test
Loading