From 83e8da95c1c6eeda0a9fe2881b18edb5f9fe3008 Mon Sep 17 00:00:00 2001 From: Jonathan Turner Date: Tue, 30 Mar 2021 12:52:16 +1300 Subject: [PATCH 1/6] Add the proposal for shipping 1.0 --- text/0006-path-to-1.0.md | 188 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 text/0006-path-to-1.0.md diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md new file mode 100644 index 0000000..49767b4 --- /dev/null +++ b/text/0006-path-to-1.0.md @@ -0,0 +1,188 @@ +- Feature Name: path_to_1_0 +- Start Date: 2021-03-30 +- RFC PR: [nushell/rfcs#0006](https://github.com/nushell/rfcs/pull/0006) + +# Summary +[summary]: #summary + +This document is the proposal to help direct Nushell towards a +successful 1.0 release. It's loosely divided into sections based on +what needs to happen first, followed by what needs to happen next, and +so on. + +# Motivation +[motivation]: #motivation + +As both a language and a tool, Nushell would benefit both by hitting 1.0 and by reaching 1.0 at the highest possible quality in a reasonable amount of time. + +This allow potential users who have waited for Nushell to reach a stable release to start using Nushell. It also allow developers to create Nushell scripts with confidence that these scripts will continue working in the future. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +We'd like to follow in the steps of Rust. As it came time to pick what features would be part of Rust's 1.0 release, the Rust leadership put features into two buckets: those not ready for 1.0 and those ready for 1.0. All features started in the "not ready for 1.0" bucket and had to prove they were stable, supportable, and worked together with the other features that had made it into 1.0. + +Nushell can follow the same path. All Nushell features would start in the "not ready to ship in 1.0" bucket and each one would have to prove that it was ready for 1.0. + +The lists below show features we hope will make it for 1.0. It may also be possible that some features become ready that we are not currently anticipating, while other features we hoped would ship do no ultimately meet the bar. + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## 0.6 milestone + +The most important things to get designed and implemented first are +the changes which will be the most disruptive to current users of +Nushell. We're aware that any change at this point needs to be +well-considered and also needs to land sooner rather than later. The +longer we wait, the harder it will be for users of Nushell to absorb +the change into their growing collection of scripts and shortcuts. + +### Elevator pitch + +Many of us appreciate different aspects of Nushell, but one aspect we +can improve is how to share that succinctly with the rest of the +world. Nushell represents some interesting choices that help improve +developer workflows, make data processing easier, and also offers +dozens of opportunities for more easily working with your system. + +We'd like to develop a way to describe the above, and more, in a way +that's easy for potential users of Nushell to grasp why Nushell is +important and how it could fit their needs. + +### Solidifying core syntax + +There are a set of syntax that are used often and that will cause +breakage broadly if changed. Solidifying this syntax also allows us to +more securely build on top of it for 1.0 features. + +Here are some of the syntax points that would be good to solidify: + +**Variables and expressions** + +* Variables currently use `$` sometimes and not others. We should + settle on when and why. +* Expressions invocations currently use `$()`. We should explore `()` + as a possible simpler syntax. +* Clarify difference between `each`, `if`, and `where`. The `$it` + variable feels a bit special and also unpredictable. We need a + design that makes it easy to predict when it will be available and + what it will do. +* A possible new `$in` variable, or related, to represent the input to + a command +* A yield or `$out` style variable to allow multiple lines to + participate in output in a command + +**Additional syntax** + +* Doc comments: what are they and where can we use them +* Blocks have parameters but there is no syntax for them, yet + +### Core data model + +* Streams as values. Will it be possible? If so, how will it work? + +### Types + +* Future-proof type syntax +* Enable building support for external command autocomplete + +### Philosophy + +* Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. +* Should we raise exceptions if there are holes in the data as you + process it? When and how? + + +## 0.8 milestone + +The key part of the 0.8 milestone is to **finish language-breaking +changes**. As part of this, we'd also like to finish semantic changes +to the core commands as well, as they make up an important part of the +Nushell language. + +### Core commands + +* Clean up `str` +* Change `echo` to actually echo to the screen +* Use another function to mean "send value to pipeline" +* Final list of internal commands + +### Ergonomics + +* Improve adding paths to the PATH +* Cache config +* Set baseline performance target +* Shorthand column `get` +* Completions +* Script parameters + +### Language concepts + +* Imports and modules +* When do we iterate? Should `=` iterate? +* Can you operate on the incoming pipeline as a single value? If so, how? +* Related to the above: xargs-like functionality + +### Configuration + +* Solidify the file formats we'll use + +## 1.0 milestone + +### Future-proofing + +* Future-proof string/path interpolation +* Future-proof ability to process stream in parallel +* Scope-down and future-proof plugin protocol + +### Documentation + +* Book 1.0 +* Translations of documents + +### Ergonomics + +* Paging +* Per-session history and global history +* Polished error messages +* Fix starship issues +* Better structured data matching (like grep over structured data) + +### Final designs + +* Input/output types +* External->Internal final design +* Internal->External final design + +### Shipping quality + +* Sign binary releases +* Default install artifacts (what will Nushell include at 1.0?) +* Move unstable features behind feature flags +* Enable self-update capabilities? +* Ship wasm demo for 1.0 + +### Fixing outstanding bugs + +* Remove all dynamic scoping. Ensure all scoping is lexical. + +# Drawbacks +[drawbacks]: #drawbacks + +- One drawback is that this will mean moving from exploration energy to shipping energy. +- A related drawback is that some features may not make it and will need to be removed to ensure a high quality 1.0 release. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +- The main wiggle room is in what features we go after and what we do not. We may decide a different set of features are a better fit for 1.0 that those in this proposal. That new set would need clear rationale for why its endpoint would be stronger than this. + +# Future possibilities +[future-possibilities]: #future-possibilities + +The 1.0 release marks an important achievement for Nushell. After this point, scripts written against 1.0 will continue to work into the future. This stable language will allow people to create a growing library of scripts, share scripts with each other, and trust Nushell when doing production work. + +There are a number of potential future features that we could enable after 1.0 ships: more varied uses for Shells, a robust system for managing background tasks, type-provider style completions to name a few. + +We, as a team, take this commitment very seriously, and are looking forward to shipping a 1.0 we're proud of. If you'd like to help, please join us on the [discord](https://discord.gg/NtAbbGn). We'd be happy to have your input. \ No newline at end of file From ca2b24017dbc2ac1382630ef19c8b25bd099331e Mon Sep 17 00:00:00 2001 From: Jonathan Turner Date: Tue, 30 Mar 2021 17:54:02 +1300 Subject: [PATCH 2/6] Update 0006-path-to-1.0.md --- text/0006-path-to-1.0.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md index 49767b4..4597e2e 100644 --- a/text/0006-path-to-1.0.md +++ b/text/0006-path-to-1.0.md @@ -15,7 +15,7 @@ so on. As both a language and a tool, Nushell would benefit both by hitting 1.0 and by reaching 1.0 at the highest possible quality in a reasonable amount of time. -This allow potential users who have waited for Nushell to reach a stable release to start using Nushell. It also allow developers to create Nushell scripts with confidence that these scripts will continue working in the future. +This allows potential users who have waited for Nushell to reach a stable release to start using Nushell. It also allows developers to create Nushell scripts with confidence that these scripts will continue working in the future. # Guide-level explanation [guide-level-explanation]: #guide-level-explanation @@ -185,4 +185,4 @@ The 1.0 release marks an important achievement for Nushell. After this point, sc There are a number of potential future features that we could enable after 1.0 ships: more varied uses for Shells, a robust system for managing background tasks, type-provider style completions to name a few. -We, as a team, take this commitment very seriously, and are looking forward to shipping a 1.0 we're proud of. If you'd like to help, please join us on the [discord](https://discord.gg/NtAbbGn). We'd be happy to have your input. \ No newline at end of file +We, as a team, take this commitment very seriously, and are looking forward to shipping a 1.0 we're proud of. If you'd like to help, please join us on the [discord](https://discord.gg/NtAbbGn). We'd be happy to have your input. From 59c165583b173db60c64f13310936d2278140b37 Mon Sep 17 00:00:00 2001 From: JT Date: Wed, 2 Jun 2021 09:37:47 +1200 Subject: [PATCH 3/6] Update 0006-path-to-1.0.md --- text/0006-path-to-1.0.md | 100 +++++++++++++++++++-------------------- 1 file changed, 50 insertions(+), 50 deletions(-) diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md index 4597e2e..83f38d2 100644 --- a/text/0006-path-to-1.0.md +++ b/text/0006-path-to-1.0.md @@ -29,7 +29,7 @@ The lists below show features we hope will make it for 1.0. It may also be possi # Reference-level explanation [reference-level-explanation]: #reference-level-explanation -## 0.6 milestone +## 0.60 milestone The most important things to get designed and implemented first are the changes which will be the most disruptive to current users of @@ -60,41 +60,41 @@ Here are some of the syntax points that would be good to solidify: **Variables and expressions** -* Variables currently use `$` sometimes and not others. We should +- [x] Variables currently use `$` sometimes and not others. We should settle on when and why. -* Expressions invocations currently use `$()`. We should explore `()` +- [x] Expressions invocations currently use `$()`. We should explore `()` as a possible simpler syntax. -* Clarify difference between `each`, `if`, and `where`. The `$it` +- [ ] Clarify difference between `each`, `if`, and `where`. The `$it` variable feels a bit special and also unpredictable. We need a design that makes it easy to predict when it will be available and what it will do. -* A possible new `$in` variable, or related, to represent the input to +- [ ] A possible new `$in` variable, or related, to represent the input to a command -* A yield or `$out` style variable to allow multiple lines to +- [ ] A yield or `$out` style variable to allow multiple lines to participate in output in a command **Additional syntax** -* Doc comments: what are they and where can we use them -* Blocks have parameters but there is no syntax for them, yet +- [ ] Doc comments: what are they and where can we use them +- [x] Blocks have parameters but there is no syntax for them, yet ### Core data model -* Streams as values. Will it be possible? If so, how will it work? +- [ ] Streams as values. Will it be possible? If so, how will it work? + * eg) `ls | echo %1` + * Related to the above: xargs-like functionality ### Types -* Future-proof type syntax -* Enable building support for external command autocomplete +- [ ] Future-proof type syntax +- [ ] Enable building support for external command autocomplete ### Philosophy -* Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. -* Should we raise exceptions if there are holes in the data as you - process it? When and how? +- [ ] Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. +- [ ] Should we raise exceptions if there are holes in the data as you process it? When and how? - -## 0.8 milestone +## 0.80 milestone The key part of the 0.8 milestone is to **finish language-breaking changes**. As part of this, we'd also like to finish semantic changes @@ -103,69 +103,69 @@ Nushell language. ### Core commands -* Clean up `str` -* Change `echo` to actually echo to the screen -* Use another function to mean "send value to pipeline" -* Final list of internal commands +- [ ] Clean up `str` +- [ ] Change `echo` to actually echo to the screen +- [x] Use another function to mean "send value to pipeline" +- [ ] Explicit iteration expansion (eg, like what `echo 1..10` is today) +- [ ] Final list of internal commands ### Ergonomics -* Improve adding paths to the PATH -* Cache config -* Set baseline performance target -* Shorthand column `get` -* Completions -* Script parameters +- [ ] Improve adding paths to the PATH +- [ ] Cache config +- [ ] Set baseline performance target +- [ ] Shorthand column `get` +- [ ] Completions +- [ ] Script parameters ### Language concepts -* Imports and modules -* When do we iterate? Should `=` iterate? -* Can you operate on the incoming pipeline as a single value? If so, how? -* Related to the above: xargs-like functionality +- [ ] Imports and modules +- [ ] When do we iterate? Should `=` iterate? +- [ ] Can you operate on the incoming pipeline as a single value? If so, how? ### Configuration -* Solidify the file formats we'll use +- [ ] Solidify the file formats we'll use ## 1.0 milestone ### Future-proofing -* Future-proof string/path interpolation -* Future-proof ability to process stream in parallel -* Scope-down and future-proof plugin protocol +- [ ] Future-proof string/path interpolation +- [ ] Future-proof ability to process stream in parallel +- [ ] Scope-down and future-proof plugin protocol ### Documentation -* Book 1.0 -* Translations of documents +- [ ] Book 1.0 +- [ ] Translations of documents ### Ergonomics -* Paging -* Per-session history and global history -* Polished error messages -* Fix starship issues -* Better structured data matching (like grep over structured data) +- [ ] Paging +- [ ] Per-session history and global history +- [ ] Polished error messages +- [ ] Fix starship issues +- [ ] Better structured data matching (like grep over structured data) ### Final designs -* Input/output types -* External->Internal final design -* Internal->External final design +- [ ] Input/output types +- [ ] External->Internal final design +- [ ] Internal->External final design ### Shipping quality -* Sign binary releases -* Default install artifacts (what will Nushell include at 1.0?) -* Move unstable features behind feature flags -* Enable self-update capabilities? -* Ship wasm demo for 1.0 +- [ ] Sign binary releases +- [ ] Default install artifacts (what will Nushell include at 1.0?) +- [ ] Move unstable features behind feature flags +- [ ] Enable self-update capabilities? +- [ ] Ship wasm demo for 1.0 ### Fixing outstanding bugs -* Remove all dynamic scoping. Ensure all scoping is lexical. +- [ ] Remove all dynamic scoping. Ensure all scoping is lexical. # Drawbacks [drawbacks]: #drawbacks From 46dfa3ed3a843e9b3cd5b89c61f81e886dd34d2c Mon Sep 17 00:00:00 2001 From: JT Date: Wed, 9 Jun 2021 10:00:44 +1200 Subject: [PATCH 4/6] Update 0006-path-to-1.0.md --- text/0006-path-to-1.0.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md index 83f38d2..d871794 100644 --- a/text/0006-path-to-1.0.md +++ b/text/0006-path-to-1.0.md @@ -89,6 +89,10 @@ Here are some of the syntax points that would be good to solidify: - [ ] Future-proof type syntax - [ ] Enable building support for external command autocomplete +### Fixing outstanding bugs + +- [ ] Remove all dynamic scoping. Ensure all scoping is lexical. + ### Philosophy - [ ] Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. @@ -114,7 +118,7 @@ Nushell language. - [ ] Improve adding paths to the PATH - [ ] Cache config - [ ] Set baseline performance target -- [ ] Shorthand column `get` +- [x] Shorthand column `get` - [ ] Completions - [ ] Script parameters @@ -132,8 +136,7 @@ Nushell language. ### Future-proofing -- [ ] Future-proof string/path interpolation -- [ ] Future-proof ability to process stream in parallel +- [x] Future-proof string/path interpolation - [ ] Scope-down and future-proof plugin protocol ### Documentation @@ -146,12 +149,11 @@ Nushell language. - [ ] Paging - [ ] Per-session history and global history - [ ] Polished error messages -- [ ] Fix starship issues +- [x] Fix starship issues - [ ] Better structured data matching (like grep over structured data) ### Final designs -- [ ] Input/output types - [ ] External->Internal final design - [ ] Internal->External final design @@ -163,9 +165,11 @@ Nushell language. - [ ] Enable self-update capabilities? - [ ] Ship wasm demo for 1.0 -### Fixing outstanding bugs -- [ ] Remove all dynamic scoping. Ensure all scoping is lexical. +### Possible inclusion + +- [ ] Future-proof ability to process stream in parallel + # Drawbacks [drawbacks]: #drawbacks From 0e86d3aed4886402f04af453648ecbf2c2843d0c Mon Sep 17 00:00:00 2001 From: JT Date: Wed, 9 Jun 2021 10:38:55 +1200 Subject: [PATCH 5/6] Update 0006-path-to-1.0.md --- text/0006-path-to-1.0.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md index d871794..4b47234 100644 --- a/text/0006-path-to-1.0.md +++ b/text/0006-path-to-1.0.md @@ -83,6 +83,8 @@ Here are some of the syntax points that would be good to solidify: - [ ] Streams as values. Will it be possible? If so, how will it work? * eg) `ls | echo %1` * Related to the above: xargs-like functionality +- [ ] Explor dataframes as the transport for data inside of the engine + * eg) polars+apache arrow ### Types From 83c6d1746ac285bd68a30a1e1fb091f191ea3615 Mon Sep 17 00:00:00 2001 From: JT <547158+jntrnr@users.noreply.github.com> Date: Sat, 16 Apr 2022 18:11:32 +1200 Subject: [PATCH 6/6] Update 0006-path-to-1.0.md --- text/0006-path-to-1.0.md | 40 ++++++++++++++++++++-------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/text/0006-path-to-1.0.md b/text/0006-path-to-1.0.md index 4b47234..bfc4173 100644 --- a/text/0006-path-to-1.0.md +++ b/text/0006-path-to-1.0.md @@ -64,11 +64,11 @@ Here are some of the syntax points that would be good to solidify: settle on when and why. - [x] Expressions invocations currently use `$()`. We should explore `()` as a possible simpler syntax. -- [ ] Clarify difference between `each`, `if`, and `where`. The `$it` +- [x] Clarify difference between `each`, `if`, and `where`. The `$it` variable feels a bit special and also unpredictable. We need a design that makes it easy to predict when it will be available and what it will do. -- [ ] A possible new `$in` variable, or related, to represent the input to +- [x] A possible new `$in` variable, or related, to represent the input to a command - [ ] A yield or `$out` style variable to allow multiple lines to participate in output in a command @@ -80,24 +80,24 @@ Here are some of the syntax points that would be good to solidify: ### Core data model -- [ ] Streams as values. Will it be possible? If so, how will it work? +- [x] Streams as values. Will it be possible? If so, how will it work? (now `$in`) * eg) `ls | echo %1` * Related to the above: xargs-like functionality -- [ ] Explor dataframes as the transport for data inside of the engine +- [ ] Explore dataframes as the transport for data inside of the engine * eg) polars+apache arrow ### Types - [ ] Future-proof type syntax -- [ ] Enable building support for external command autocomplete +- [x] Enable building support for external command autocomplete ### Fixing outstanding bugs -- [ ] Remove all dynamic scoping. Ensure all scoping is lexical. +- [x] Remove all dynamic scoping. Ensure all scoping is lexical. ### Philosophy -- [ ] Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. +- [x] Are repl/script the same? Where do we differ? Related: align `;` to work the same as `\n`. - [ ] Should we raise exceptions if there are holes in the data as you process it? When and how? ## 0.80 milestone @@ -110,25 +110,25 @@ Nushell language. ### Core commands - [ ] Clean up `str` -- [ ] Change `echo` to actually echo to the screen +- [x Change `echo` to actually echo to the screen (now `print` does this) - [x] Use another function to mean "send value to pipeline" -- [ ] Explicit iteration expansion (eg, like what `echo 1..10` is today) +- [x] Explicit iteration expansion (eg, like what `echo 1..10` is today) - [ ] Final list of internal commands ### Ergonomics -- [ ] Improve adding paths to the PATH -- [ ] Cache config +- [x] Improve adding paths to the PATH +- [x] Cache config - [ ] Set baseline performance target - [x] Shorthand column `get` -- [ ] Completions -- [ ] Script parameters +- [x] Completions +- [x] Script parameters ### Language concepts -- [ ] Imports and modules -- [ ] When do we iterate? Should `=` iterate? -- [ ] Can you operate on the incoming pipeline as a single value? If so, how? +- [x] Imports and modules +- [x] When do we iterate? Should `=` iterate? +- [x] Can you operate on the incoming pipeline as a single value? If so, how? (now: `$in`) ### Configuration @@ -148,7 +148,7 @@ Nushell language. ### Ergonomics -- [ ] Paging +- [x] Paging (now: use external pagers) - [ ] Per-session history and global history - [ ] Polished error messages - [x] Fix starship issues @@ -156,8 +156,8 @@ Nushell language. ### Final designs -- [ ] External->Internal final design -- [ ] Internal->External final design +- [x] External->Internal final design +- [x] Internal->External final design ### Shipping quality @@ -170,7 +170,7 @@ Nushell language. ### Possible inclusion -- [ ] Future-proof ability to process stream in parallel +- [x] Future-proof ability to process stream in parallel # Drawbacks