Skip to content

Latest commit

 

History

History
154 lines (130 loc) · 11.1 KB

README.md

File metadata and controls

154 lines (130 loc) · 11.1 KB

@hyoban/api

A Cloudflare Worker that serves the entries, collections, places, trips' routes and the car's statistics from D1 and their media from R2, for the site's build and the Android app, which writes through it one change set at a time. Records come back as stored, in the shapes @hyoban/data-spec defines. Deployment, secrets and bindings are in docs/deploy.md.

Reading

Every request, GET / included, needs Authorization: Bearer <token> with one of the API_TOKENS (comma-separated), and then sees everything, drafts and private records included; without one, or with any other, it is a 401. The site trims what it shows to what is public itself, as getPublicEntry does.

Path Returns
GET / { name, revision, assets }, assets the origin the photos and videos are served from
GET /v2/rows?since=&limit= every row of every table, deleted ones too, for a copy of D1 row for row
GET /v2/entries/:uid one entry
GET /v2/collections, /v2/collections/:key { revision, collections } by start date, or one by its slug (else its uid)
GET /v2/export { revision, manifest, places, entries, collections, routes, vehicleMonths, vehicleAssisted, vehicleCaptures }
GET /v2/places[/:id] { revision, places } by id, or one place
GET /v2/routes { revision, trips }, each published trip's route as stretches, with an ETag

The records are entries and collections (@hyoban/data-spec/entry). Every table is one of @hyoban/data-spec/tables (schema.sql, generated by vp run data:tables, which the migrations must end at): a key, the revision it last changed at, when it was deleted and the record's JSON, with the columns SQLite works out from the record. The app's copy keeps the same tables.

The rows

Every write bumps the database's revision, and each row keeps the revision it last changed at. GET /v2/rows?since=<revision> hands out the rows that changed after it, deleted ones too, limit at a time (100 by default, at most 500): each { table, key, rev, deletedAt, record } of the tables in @hyoban/data-spec/tables but the routes (served as GET /v2/routes), a table at a time in its order, places before the entries that name them, all of them read in one D1 batch. Follow next with ?cursor=<next> until it is null, then keep revision for the next since. Every page is of the revision the first was read at; a row changed since is left for the next read. database says which database the rows are of (meta's database, a number at random): a copy that kept another reads every row again. since=0 is every row; nothing is dropped from the database, only marked deleted, so the app can keep the same tables D1 does.

Writing

Path Does
PUT /v2/entries/:uid/media/:mediaUid/:part keeps an original, image or video, until its entry publishes; 422 for an image that says where it was taken
HEAD on the same path 200 with the original's MD5 as its ETag once uploaded, else 404
POST /v2/changes applies { places, entries, deleted, collections, deletedCollections } as one write; an imported charge is refused
POST /v2/build calls the site's deploy hook and posts to the channel; { started }
POST /v2/routes fetches missing routes; { fetched, removed, failed, left }

A change set is checked whole, then written in one transaction that bumps the revision and stamps the manifest's updatedAt; one that doesn't fit changes nothing (a 422 names every missing original), and one that changes nothing takes no revision. The answer gives the id each place is known by, the records as published, the uids deleted and the revision taken.

  • Places. A place keeps the id it is sent with, the app's p-<ulid> included, so its bills, check-ins and Moments never need moving: a known id is that place, and any other is added. Places near each other, of one name or of one Swarm venue stay apart; choosing a known one is the app's to do as the place is picked. A new place has its province, city and district looked up from DataV's GeoAtlas (DIVISIONS_URL), or left for vp run places:locate when that fails. A known place sent under its own id takes the name, point and kind sent with it and keeps the rest; moved, its division is looked up again and the trips' routes are worked out again. An entry names one place at most.
  • Publishing. A draft takes a published block with the time, and its originals move to moments/<uid>/<media uid>.<ext> (getMediaKey). The published blocks are the server's and never change. Media are kept as uploaded (original: true), and the site has Cloudflare resize them (docs/assets.md). An original is public at its key, so it must not say where it was taken: an image whose EXIF or XMP has a GPS position is turned down (@hyoban/data-spec/location); a video is streamed through unread, and the app takes its location out before sending it. The media types it takes are @hyoban/data-spec/publish's.
  • Privacy. Of a plugin that keeps to the phone (a bill), a private part is dropped even when sent, and a public one is kept by its public fields alone (getUploadEntry).
  • After a write the site shows, the trips' routes are brought up to date when what they are built from changed, then DEPLOY_HOOK_URL is called so the site builds again.

Records written outside the API go in with vp run data:push (docs/data.md), which then calls POST /v2/build: the site builds again and new Moments post to the channel.

Routes

Each leg between consecutive places of a trip (its Moments, its marks and its public check-ins, as @hyoban/trip-map's buildTrip and the spec's collectionToTrip work them out) has a route in the routes table by its key: a drive from FOSSGIS's public Valhalla, less the spur it drives on and back at either end to reach a place across a divided road (trip-map's trimTurnBacks, its distance and time shortened by as much), or for a train leg (train: key) a rail line from OpenRailRouting's demo between the places' stations. Neither needs a key, so the app can route the same way. Ferries and flights are straight. After a write the trips see, the Worker fetches the legs it lacks (up to 30, 2 at once, as the public routers ask) and marks deleted those no trip has. A leg with no way along it is remembered and drawn straight; a failed fetch is retried after the next write. POST /v2/routes fills the same way on request; POST /v2/routes?again=drives first queues every drive to be routed again, as after a change of router, and each fill then takes its batch from the queue after the missing legs, each leg keeping its old route until the new one comes.

GET /v2/routes gives each published trip's route as the site's overview draws it, drives joined into one stretch and other legs dashed. The answer is cached in meta with its revision, and its ETag covers the routes alone, so If-None-Match gets a 304 when a write changed nothing drawn:

{ "revision": 12, "trips": { "2026-yunnan": [{ "line": [[100.2, 26.8], …], "straight": false }] } }

The Telegram channel

With TELEGRAM_BOT_TOKEN, the Worker posts the Moments the site shows to the channel TELEGRAM_CHAT_ID names, through a bot that is an admin allowed to post and delete: the words exactly, a line of 📍 and the place, 🕐 and when it was at its own clock (the year only in another year), 🧳 and the trip it was on unless the trip is private, with 💰 人均 for what it cost and how it was (😋 好吃, 😌 还行 or 😖 踩雷 for a meal, else 👍 值, 👌 还行 or 👎 不值), with no link, as the caption of its photos and videos in albums of ten, or after them past 1024 characters. Photos go as JPEGs from Cloudflare Images; a video over 50 MB goes as its first frame.

Changed words edit the post, and changed photos or videos are replaced in their messages; media that change in number repost it, as Telegram can't add to an album or take from it. A Moment the site stops showing has its post deleted, and so does one made planned: a planned Moment is posted only once it is made a real one. telegram_posts keeps each Moment's messages. Only Moments published since telegram_since are posted, so a restore never reposts old ones. It runs after a write the site shows, after POST /v2/build, and on the 15-minute cron, from the revision the last whole run reached (telegram_revision), under a lease in meta so two runs never post twice.

Setting up

vp run data:import            # migrations and data/ into the local database
vp run --filter @hyoban/api dev

Put API_TOKENS=<token> in apps/api/.dev.vars: the local API, like the deployed one, answers nothing without it. data:import needs an exported data/ (see docs/data.md).

Layout

  • migrations/: the D1 schema. Each table keeps the record as written in a record column, so fields the spec doesn't know yet survive, plus the columns it is looked up by.
  • src/index.ts routes requests, src/auth.ts checks their token and src/http.ts shapes the responses.
  • src/database.ts is the few D1 calls the rest needs (Database) and the places' reads; src/bucket.ts the R2 ones (Bucket). The tests fill them in with an in-memory node:sqlite and a map.
  • src/entries.ts reads the entries and collections, writes /v2/export and applies a change set; src/writes.ts uploads and publishes media, names places and commits.
  • src/copy.ts serves /v2/rows, src/routes.ts the trips' routes and src/site.ts the site's deploy hook.
  • src/telegram.ts posts to the channel, against a Telegram the tests fake.
  • src/rows.ts turns a record into a row, for the writes and the import.
  • src/import.ts builds the SQL vp run data:import runs: new and changed records written, missing ones marked deleted, unchanged rows keeping their revision.