slugify replaceable?

[flicksolutions]

Could the slugify function be replaced by encodeURIComponent?

It seems to me that it does roughly the same.

[flicksolutions]

@vvvyyynet

[vvvyyynet]

Good point. What I don't like about encodeURIComponent however, is that it produces highly non-human-readable text, which is a pitty I think.

I just did some research and found this article that promotes this more universal solution for slug creation. I think it's quite nice:

const removeDiacritics = (s) => {
  return s
    .normalize('NFD')
    .replace(/[\u0300-\u036f]/g, '')
}

const generateSlug = (title) => {
  return removeDiacritics(title)
    .toLowerCase()
    .replace(/[^a-z0-9\s-]/g, '')
    .replace(/\s+/g, '-')
    .replace(/-+/g, '-')
    .trim()
}

// Usage
console.log(generateSlug('Příliš žluťoučký kůň')) // 'prilis-zlutoucky-kun'

[flicksolutions]

it produces highly non-human-readable text

actually, with modern browsers it does not.
The strin 'Militärische Anlage' is transformed to a string with escape sequences, but browsers know how to handle that! Firefox translates it back to 'Militärische Anlage', chromium Browsers get 'Militärische%20Anlage' still not too bad.

.normalize('NFD').replace(/[\u0300-\u036f]/g, '') normalizes characters to their base form which means a loss of meaning. There is a category Pässe which would become Passe. which is another word in German (Passe as in Durchgang, Umlauf) so in theory you could have two identical slugs if you have a minimal Pair with Umlaut. .normalize('NFD').replace(/[\u0300-\u036f]/g, 'e') would help you with the umlaut but if you have other diacritica from other languages it wouldn't work.

I agree though, that a space (or &20) in your slug is suboptimal, but that could be solved by replacing the spaces with a - before encoding the URIComponent:
encodeURIComponent(groupLabel.replaceAll(' ', '-'))

This is very short, concise and the safest way possible in my opinion.

[pdaengeli]

Sorry for chiming in without having the full context. On what level are these slugs used?

If they are used in URLs that point to an "item" view (e.g. document, entity, about page), I would argue that stability matters before anything else. Using any library poses more risk in this regard than hard-coding the slug on the data level. We can easily add a <title type="slug"> or <idno type="slug"> to anything that needs it and expose it in all endpoints where it matters. This would give full control to the editors and allow for nuances. The only thing needed is awareness that this is to be set once and then left alone (so it would need some thinking and decisions, but in this project that's very realistic).

Or do we just use the item ID here? That is the most simple and stable approach.

If it applies to lower level things that need an address, but are not items on data level (e.g. a lemma of a commentary), hard-coding is of course over the top (and clashes might be hard to avoid). For this, I would pick a simple approach as suggested by @flicksolutions.

[vvvyyynet]

If they are used in URLs that point to an "item" view...

No, slugify.ts is currently only used to create #anchors in the URL, e.g. to jump in the Register to e.g. "Pässe". So it's not that much of a stability issue right now, but I agree 100% on your warning, @pdaengeli , wrt. routing.

There is a category Pässe which would become Passe

Good catch! Actually in the original version of slugify.ts it would convert "Pässe" to /edition/register/places#Paesse... but I agree that it's a brittle solution.

[ ...] that could be solved by replacing the spaces with a - before encoding the URIComponent encodeURIComponent(groupLabel.replaceAll(' ', '-'))

Ok, this is already much better. However, there's also a log of slugs with Gendermarkers (e.g. "Künstler:innen") as well as some with brackets (e.g. "Historische Personen (ohne persoenliche Bekanntschaft)"). So we should replace those as well I think... and suddenly we will find ourselves to rewrite the same slugify-function there was at the beginning ^^

hard-coding is of course over the top.

Indeed, I did consider hard-coding them myself. The slugs could live alongside the label_singular and label_plural in dict_register.json for group names and alongside name and preamble in doc_sequences.json for sequence names. If some new group or sequence enters the scene, of course there must be a way to fill also not forget to give it a name. Maybe, @pdaengeli, you can already have a quick look at dict_register.json and doc_sequences.json and think about how the pipeline must be set up, such that all those labels and names arrive at the Frontend (since for plural and singular forms etc.) we definitely should stick to hardcoding, I believe.

[pdaengeli]

Thanks.

However, there's also a log of slugs with Gendermarkers (e.g. "Künstler:innen") as well as some with brackets (e.g. "Historische Personen (ohne persoenliche Bekanntschaft)"). So we should replace those as well I think... and suddenly we will find ourselves to rewrite the same slugify-function there was at the beginning ^^

If the implementation is simple enough, I see some value of project specific functions over external dependencies (the Endings project is very strict about reducing the dependency footprint with only few exceptions). Here it could be

function slugify(value) {
  return value
    .normalize('NFKD')
    .replace(/[\u0300-\u036f]/g, '')      // remove accents
    .replace(/[^a-zA-Z0-9]+/g, '-')       // punctuation/whitespace -> dash
    .replace(/^-+|-+$/g, '')              // trim dashes
    .toLowerCase();
}

In the data pipelines I mostly do it that way (just using XPath).

[flicksolutions]

please don't overengineer this. The whole point of opening this ticket is, that I think the slugify function is overengineered. This slug is never displayed anywhere but in the URL bar. it's not a terrible thing if you have Künstler%3Ainnen there. and encodeURIComponent just won't break and won't create any errors downstream...