Reference changed file name in the command

[kevinburke]

Hi Jeff,
I'm writing a thing that runs Node.js tests faster. Currently we pay a ~6-10s penalty to run a single test, and the majority of this time is spent requireing a shitload of files.

Instead I'm writing a server which loads every dependency at boot. You can POST to it to invalidate the cache for a specific file, or to start a test run. It's looking pretty good so far, test runs complete with about 10ms of overhead vs 6-10s. I have a companion command line tool for the server.

What I want is to write a watcher that listens for changes to any file in the project, then invalidates the cache for that file, e.g.

1. Detect change to api/models/Users
2. Run lucifer invalidate api/models/Users to clear the cache for it.

Unfortunately it seems like justrun only runs statically defined commands. I was wondering how open you'd be to defining some kind of dynamic "file changed" syntax for the passed in command, eg find api -type f | justrun -stdin -c 'lucifer invalidate {{ file }}' or similar.

If I'm reading the code correctly, it wouldn't exactly be trivial. you'd have to modify the cmdCh channel to pass the changed file path as well as a time.Time object, then do some kind of interpolation in the for loop in main.go and pass the result to cmd.Start.

I'm happy to take a crack at it, but wanted to know if you were open to the idea, or had better thoughts about the interpolation syntax.

[kevinburke]

Here's a really hacky prototype, with {{ .File }} being interpolated.. kevinburke@45a1bbc

Could clean this up and submit a proper PR

[jmhodges]

Okay! Couple of things:

First, what happens if the event gets ignored because lastStartTime was too recent (the ignoring happens in main, not watch, which is probably why you didn't notice it. Whoops)? Do we add it to the next run, maybe? If so, space or comma-separated?

Second, might be better as an environment variable since we assume bash, anyway.

I'm worried about the first one not going well but you might have some good ideas that I do not!

[kevinburke]

Second, might be better as an environment variable since we assume bash, anyway.

Yep this is a much better idea than template interpolation. Not sure what I was thinking.

First, what happens if the event gets ignored because lastStartTime was too recent (the ignoring happens in main, not watch, which is probably why you didn't notice it. Whoops)? Do we add it to the next run, maybe? If so, space or comma-separated?

Yeah, I totally missed this because I was editing one file at a time. I looked around for ways to encode an array of filenames as an environment variable and it's not very promising; we could use a comma or a space but parsing would fail if filenames contained those characters.

It seems like the only characters you can't use in a filename are a /, which we can't use because the changed file might be in a subdirectory, or the null byte \0. We could use the null byte - I haven't seen that too often in an environment variable but people might be familiar enough with it because of the find -print0 | xargs -0 pattern. Need to figure out if you can reliably encode/read a null byte into an environment variable.

The other option would be to use an encoding that's more complicated than a single delimiter character, like CSV or JSON. Might make parsing too complex.