# https://nushell.sh/cookbook/ llms-full.txt <|firecrawl-page-1-lllmstxt|> ## Git Log Parsing Guide # [Parsing Git Log](https://www.nushell.sh/cookbook/parsing_git_log.html\#parsing-git-log) # [Let's parse git log](https://www.nushell.sh/cookbook/parsing_git_log.html\#let-s-parse-git-log) This `git log` command is interesting but you can't do a lot with it like this. ```nu git log ``` Let's make it more parsable ```nu git log --pretty="%h|%s|%aN|%aE|%aD" -n 25 ``` This will work but I've been burnt by this in the past when a pipe `|` gets injected in the commits. So, let's try again with something that most likely won't show up in commits, `»¦«`. Also, since we're not using a pipe now we don't have to use quotes around the pretty format string. Notice that the output is just a bunch of strings. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 # => 42f1874a»¦«Update some examples and docs (#4682)»¦«Justin Ma»¦«hustcer@outlook.com»¦«Tue, 1 Mar 2022 21:05:29 +0800 # => 2a89936b»¦«Move to latest stable crossterm, with fix (#4684)»¦«Sophia»¦«547158+sophiajt@users.noreply.github.com»¦«Tue, 1 Mar 2022 07:05:46 -0500 # => ece5e7db»¦«dataframe list command (#4681)»¦«Fernando Herrera»¦«fernando.j.herrera@gmail.com»¦«Tue, 1 Mar 2022 11:41:13 +0000 # => a6a96b29»¦«Add binary literals (#4680)»¦«Sophia»¦«547158+sophiajt@users.noreply.github.com»¦«Mon, 28 Feb 2022 18:31:53 -0500 # => e3100e6a»¦«Fix alias in `docs/sample_config/config.toml` (#4669)»¦«Luca Trevisani»¦«lucatrv@hotmail.com»¦«Mon, 28 Feb 2022 22:47:14 +0100 ``` Ahh, much better. Now that we have the raw data, let's try to parse it with nu. First we need to get it in lines or rows. Notice that the output is now in a table format. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 | lines # => ───┬───────────────────────────────────────────────────────────────────────────────────────────────── # => 0 │ 42f1874a»¦«Update some examples and docs (#4682)»¦«Justin Ma»¦«hustcer@outlook.com»¦«Tue, 1 Mar # => │ 2022 21:05:29 +0800 # => 1 │ 2a89936b»¦«Move to latest stable crossterm, with fix # => │ (#4684)»¦«Sophia»¦«547158+sophiajt@users.noreply.github.com»¦«Tue, 1 Mar 2022 07:05:46 -0500 # => 2 │ ece5e7db»¦«dataframe list command (#4681)»¦«Fernando # => │ Herrera»¦«fernando.j.herrera@gmail.com»¦«Tue, 1 Mar 2022 11:41:13 +0000 # => 3 │ a6a96b29»¦«Add binary literals (#4680)»¦«Sophia»¦«547158+sophiajt@users.noreply.github.com»¦«Mon, 28 # => │ Feb 2022 18:31:53 -0500 # => 4 │ e3100e6a»¦«Fix alias in `docs/sample_config/config.toml` (#4669)»¦«Luca # => │ Trevisani»¦«lucatrv@hotmail.com»¦«Mon, 28 Feb 2022 22:47:14 +0100 # => ───┴───────────────────────────────────────────────────────────────────────────────────────────────── ``` That's more like nushell, but it would be nice to have some columns. We used the delimiter `»¦«` specifically so we can create columns so let's use it like this. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 | lines | split column "»¦«" # => ───┬──────────┬──────────────────────┬──────────────────┬────────────────────────┬────────────────── # => # │ column1 │ column2 │ column3 │ column4 │ column5 # => ───┼──────────┼──────────────────────┼──────────────────┼────────────────────────┼────────────────── # => 0 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ Tue, 1 Mar 2022 # => │ │ and docs (#4682) │ │ │ 21:05:29 +0800 # => 1 │ 2a89936b │ Move to latest │ Sophia │ 547158+sophiajt@users. │ Tue, 1 Mar 2022 # => │ │ stable crossterm, │ │ noreply.github.com │ 07:05:46 -0500 # => │ │ with fix (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list │ Fernando Herrera │ fernando.j.herrera@g │ Tue, 1 Mar 2022 # => │ │ command (#4681) │ │ mail.com │ 11:41:13 +0000 # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users. │ Mon, 28 Feb 2022 # => │ │ (#4680) │ │ noreply.github.com │ 18:31:53 -0500 # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ Mon, 28 Feb 2022 # => │ │ `docs/sample_config/ │ │ │ 22:47:14 +0100 # => │ │ config.toml` │ │ │ # => │ │ (#4669) │ │ │ # => ───┴──────────┴──────────────────────┴──────────────────┴────────────────────────┴────────────────── ``` Yay, for columns! But wait, it would really be nice if those columns had something other than generically named column names. Let's try adding the columns names to `split column` like this. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 | lines | split column "»¦«" commit subject name email date # => ───┬──────────┬──────────────────────┬──────────────────┬────────────────────────┬────────────────── # => # │ commit │ subject │ name │ email │ date # => ───┼──────────┼──────────────────────┼──────────────────┼────────────────────────┼────────────────── # => 0 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ Tue, 1 Mar 2022 # => │ │ and docs (#4682) │ │ │ 21:05:29 +0800 # => 1 │ 2a89936b │ Move to latest │ Sophia │ 547158+sophiajt@users. │ Tue, 1 Mar 2022 # => │ │ stable crossterm, │ │ noreply.github.com │ 07:05:46 -0500 # => │ │ with fix (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list │ Fernando Herrera │ fernando.j.herrera@g │ Tue, 1 Mar 2022 # => │ │ command (#4681) │ │ mail.com │ 11:41:13 +0000 # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users. │ Mon, 28 Feb 2022 # => │ │ (#4680) │ │ noreply.github.com │ 18:31:53 -0500 # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ Mon, 28 Feb 2022 # => │ │ `docs/sample_config/ │ │ │ 22:47:14 +0100 # => │ │ config.toml` │ │ │ # => │ │ (#4669) │ │ │ # => ───┴──────────┴──────────────────────┴──────────────────┴────────────────────────┴────────────────── ``` Ahhh, that looks much better. Hmmm, that date string is a string. If it were a date vs a string it could be used for sorting by date. The way we do that is we have to convert the datetime to a real datetime and update the column. Try this. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} # => ───┬──────────┬──────────────────────────┬──────────────────┬────────────────────────────┬────────────── # => # │ commit │ subject │ name │ email │ date # => ───┼──────────┼──────────────────────────┼──────────────────┼────────────────────────────┼────────────── # => 0 │ 42f1874a │ Update some examples and │ Justin Ma │ hustcer@outlook.com │ 7 hours ago # => │ │ docs (#4682) │ │ │ # => 1 │ 2a89936b │ Move to latest stable │ Sophia │ 547158+sophiajt@users.nore │ 8 hours ago # => │ │ crossterm, with fix │ │ ply.github.com │ # => │ │ (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list command │ Fernando Herrera │ fernando.j.herrera@gmail │ 8 hours ago # => │ │ (#4681) │ │ .com │ # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users.nore │ 20 hours ago # => │ │ (#4680) │ │ ply.github.com │ # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ a day ago # => │ │ `docs/sample_config/conf │ │ │ # => │ │ ig.toml` │ │ │ # => │ │ (#4669) │ │ │ # => ───┴──────────┴──────────────────────────┴──────────────────┴────────────────────────────┴────────────── ``` Now this looks more nu-ish If we want to revert back to a date string we can do something like this with the `nth` command and the `get` command. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 5 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | select 3 | get date | format date | get 0 # => Mon, 28 Feb 2022 18:31:53 -0500 ``` Cool! Now that we have a real datetime we can do some interesting things with it like `group-by` or `sort-by` or `where`. Let's try `sort-by` first ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | sort-by date # => ────┬──────────┬──────────────────────────┬───────────────────┬───────────────────────────┬────────────── # => # │ commit │ subject │ name │ email │ date # => ────┼──────────┼──────────────────────────┼───────────────────┼───────────────────────────┼────────────── # => 0 │ 0c3ea636 │ Add support for stderr │ Sophia │ 547158+sophiajt@users.nor │ 4 days ago # => │ │ and exit code (#4647) │ │ eply.github.com │ # => 1 │ ed46f0ea │ fix: add missing │ Jae-Heon Ji │ 32578710+jaeheonji@user │ 3 days ago # => │ │ metadata for `ls_colors` │ │ s.noreply.github.com │ # => │ │ (#4603) │ │ │ # => 2 │ 3eca43c0 │ Plugins without file │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4650) │ │ l.com │ # => 3 │ 11bc0565 │ Find with regex flag │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4649) │ │ l.com │ # => 4 │ d2bd71d2 │ add LAST_EXIT_CODE │ LordMZTE │ lord@mzte.de │ 3 days ago # => │ │ variable (#4655) │ │ │ # => 5 │ 799fa984 │ Update reedline, revert │ Stefan Holderbach │ sholderbach@users.norep │ 3 days ago # => │ │ crossterm (#4657) │ │ ly.github.com │ # => 6 │ 995757c0 │ flags for find (#4663) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 7 │ 446c2aab │ Lets internals also │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ have exit codes (#4664) │ │ eply.github.com │ # => 8 │ 10ceac99 │ menu keybindings in │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ default file (#4651) │ │ l.com │ # => 9 │ 4ebbe07d │ Polars upgrade (#4665) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 10 │ 78192100 │ Add shortcircuiting │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ boolean operators │ │ eply.github.com │ # => │ │ (#4668) │ │ │ # => 11 │ 796d4920 │ add char separators │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ (#4667) │ │ reply.github.com │ # => 12 │ 0f437589 │ add last exit code to │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ starship parameters │ │ reply.github.com │ # => │ │ (#4670) │ │ │ # => 13 │ ef70c8db │ Date parse refactor │ Jonathan Moore │ jtm170330@utdallas.edu │ 2 days ago # => │ │ (#4661) │ │ │ # => 14 │ 10364c4f │ don't use table │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ compaction in to nuon if │ │ eply.github.com │ # => │ │ not a table (#4671) │ │ │ # => 15 │ eec17304 │ Add profiling build │ Stefan Holderbach │ sholderbach@users.norep │ a day ago # => │ │ profile and symbol strip │ │ ly.github.com │ # => │ │ (#4630) │ │ │ # => 16 │ d6a6c4b0 │ Add back in default │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ keybindings (#4673) │ │ eply.github.com │ # => 17 │ 0924975b │ Use default_config.nu │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ by default (#4675) │ │ eply.github.com │ # => 18 │ b09acdb7 │ Fix unsupported type │ Justin Ma │ hustcer@outlook.com │ a day ago # => │ │ message for some math │ │ │ # => │ │ related commands (#4672) │ │ │ # => 19 │ cb5c61d2 │ Fix open ended ranges │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ (#4677) │ │ eply.github.com │ # => 20 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ a day ago # => │ │ `docs/sample_config/con │ │ │ # => │ │ fig.toml` │ │ │ # => │ │ (#4669) │ │ │ # => 21 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users.nor │ 20 hours ago # => │ │ (#4680) │ │ eply.github.com │ # => 22 │ ece5e7db │ dataframe list command │ Fernando Herrera │ fernando.j.herrera@gmai │ 8 hours ago # => │ │ (#4681) │ │ l.com │ # => 23 │ 2a89936b │ Move to latest stable │ Sophia │ 547158+sophiajt@users.nor │ 8 hours ago # => │ │ crossterm, with fix │ │ eply.github.com │ # => │ │ (#4684) │ │ │ # => 24 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ 7 hours ago # => │ │ and docs (#4682) │ │ │ # => ────┴──────────┴──────────────────────────┴───────────────────┴───────────────────────────┴────────────── ``` That's neat but what if I want it sorted in the opposite order? Try the `reverse` command and notice the newest commits are at the top. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | sort-by date | reverse # => ────┬──────────┬──────────────────────────┬───────────────────┬───────────────────────────┬────────────── # => # │ commit │ subject │ name │ email │ date # => ────┼──────────┼──────────────────────────┼───────────────────┼───────────────────────────┼────────────── # => 0 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ 7 hours ago # => │ │ and docs (#4682) │ │ │ # => 1 │ 2a89936b │ Move to latest stable │ Sophia │ 547158+sophiajt@users.nor │ 8 hours ago # => │ │ crossterm, with fix │ │ eply.github.com │ # => │ │ (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list command │ Fernando Herrera │ fernando.j.herrera@gmai │ 8 hours ago # => │ │ (#4681) │ │ l.com │ # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users.nor │ 20 hours ago # => │ │ (#4680) │ │ eply.github.com │ # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ a day ago # => │ │ `docs/sample_config/con │ │ │ # => │ │ fig.toml` │ │ │ # => │ │ (#4669) │ │ │ # => 5 │ cb5c61d2 │ Fix open ended ranges │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ (#4677) │ │ eply.github.com │ # => 6 │ b09acdb7 │ Fix unsupported type │ Justin Ma │ hustcer@outlook.com │ a day ago # => │ │ message for some math │ │ │ # => │ │ related commands (#4672) │ │ │ # => 7 │ 0924975b │ Use default_config.nu │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ by default (#4675) │ │ eply.github.com │ # => 8 │ d6a6c4b0 │ Add back in default │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ keybindings (#4673) │ │ eply.github.com │ # => 9 │ eec17304 │ Add profiling build │ Stefan Holderbach │ sholderbach@users.norep │ a day ago # => │ │ profile and symbol strip │ │ ly.github.com │ # => │ │ (#4630) │ │ │ # => 10 │ 10364c4f │ don't use table │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ compaction in to nuon if │ │ eply.github.com │ # => │ │ not a table (#4671) │ │ │ # => 11 │ ef70c8db │ Date parse refactor │ Jonathan Moore │ jtm170330@utdallas.edu │ 2 days ago # => │ │ (#4661) │ │ │ # => 12 │ 0f437589 │ add last exit code to │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ starship parameters │ │ reply.github.com │ # => │ │ (#4670) │ │ │ # => 13 │ 796d4920 │ add char separators │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ (#4667) │ │ reply.github.com │ # => 14 │ 78192100 │ Add shortcircuiting │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ boolean operators │ │ eply.github.com │ # => │ │ (#4668) │ │ │ # => 15 │ 4ebbe07d │ Polars upgrade (#4665) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 16 │ 10ceac99 │ menu keybindings in │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ default file (#4651) │ │ l.com │ # => 17 │ 446c2aab │ Lets internals also │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ have exit codes (#4664) │ │ eply.github.com │ # => 18 │ 995757c0 │ flags for find (#4663) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 19 │ 799fa984 │ Update reedline, revert │ Stefan Holderbach │ sholderbach@users.norep │ 3 days ago # => │ │ crossterm (#4657) │ │ ly.github.com │ # => 20 │ d2bd71d2 │ add LAST_EXIT_CODE │ LordMZTE │ lord@mzte.de │ 3 days ago # => │ │ variable (#4655) │ │ │ # => 21 │ 11bc0565 │ Find with regex flag │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4649) │ │ l.com │ # => 22 │ 3eca43c0 │ Plugins without file │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4650) │ │ l.com │ # => 23 │ ed46f0ea │ fix: add missing │ Jae-Heon Ji │ 32578710+jaeheonji@user │ 3 days ago # => │ │ metadata for `ls_colors` │ │ s.noreply.github.com │ # => │ │ (#4603) │ │ │ # => 24 │ 0c3ea636 │ Add support for stderr │ Sophia │ 547158+sophiajt@users.nor │ 4 days ago # => │ │ and exit code (#4647) │ │ eply.github.com │ # => ────┴──────────┴──────────────────────────┴───────────────────┴─────────────────────────┴────────────── ``` Now let's try `group-by` and see what happens. This is a tiny bit tricky because dates are tricky. When you use `group-by` on dates you have to remember to use the `group-by date` subcommand so it's `group-by date date_column_name`. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime | format date '%Y-%m-%d'} | group-by date # => ────────────┬──────────────── # => 2022-03-01 │ [table 3 rows] # => 2022-02-28 │ [table 8 rows] # => 2022-02-27 │ [table 8 rows] # => 2022-02-26 │ [table 5 rows] # => 2022-02-25 │ [table 1 row] # => ────────────┴──────────────── ``` This would look better if we transpose the data and name the columns ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime | format date '%Y-%m-%d'} | group-by date | transpose date count # => ───┬────────────┬──────────────── # => # │ date │ count # => ───┼────────────┼──────────────── # => 0 │ 2022-03-01 │ [table 3 rows] # => 1 │ 2022-02-28 │ [table 8 rows] # => 2 │ 2022-02-27 │ [table 8 rows] # => 3 │ 2022-02-26 │ [table 5 rows] # => 4 │ 2022-02-25 │ [table 1 row] # => ───┴────────────┴──────────────── ``` How about `where` now? Show only the records that are less than a year old. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) # => ────┬──────────┬──────────────────────────┬───────────────────┬─────────────────────────┬────────────── # => # │ commit │ subject │ name │ email │ date # => ────┼──────────┼──────────────────────────┼───────────────────┼─────────────────────────┼────────────── # => 0 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ 7 hours ago # => │ │ and docs (#4682) │ │ │ # => 1 │ 2a89936b │ Move to latest stable │ Sophia │ 547158+sophiajt@users.nor │ 8 hours ago # => │ │ crossterm, with fix │ │ eply.github.com │ # => │ │ (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list command │ Fernando Herrera │ fernando.j.herrera@gmai │ 8 hours ago # => │ │ (#4681) │ │ l.com │ # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users.nor │ 21 hours ago # => │ │ (#4680) │ │ eply.github.com │ # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ a day ago # => │ │ `docs/sample_config/con │ │ │ # => │ │ fig.toml` │ │ │ # => │ │ (#4669) │ │ │ # => 5 │ cb5c61d2 │ Fix open ended ranges │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ (#4677) │ │ eply.github.com │ # => 6 │ b09acdb7 │ Fix unsupported type │ Justin Ma │ hustcer@outlook.com │ a day ago # => │ │ message for some math │ │ │ # => │ │ related commands (#4672) │ │ │ # => 7 │ 0924975b │ Use default_config.nu │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ by default (#4675) │ │ eply.github.com │ # => 8 │ d6a6c4b0 │ Add back in default │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ keybindings (#4673) │ │ eply.github.com │ # => 9 │ eec17304 │ Add profiling build │ Stefan Holderbach │ sholderbach@users.norep │ a day ago # => │ │ profile and symbol strip │ │ ly.github.com │ # => │ │ (#4630) │ │ │ # => 10 │ 10364c4f │ don't use table │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ compaction in to nuon if │ │ eply.github.com │ # => │ │ not a table (#4671) │ │ │ # => 11 │ ef70c8db │ Date parse refactor │ Jonathan Moore │ jtm170330@utdallas.edu │ 2 days ago # => │ │ (#4661) │ │ │ # => 12 │ 0f437589 │ add last exit code to │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ starship parameters │ │ reply.github.com │ # => │ │ (#4670) │ │ │ # => 13 │ 796d4920 │ add char separators │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ (#4667) │ │ reply.github.com │ # => 14 │ 78192100 │ Add shortcircuiting │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ boolean operators │ │ eply.github.com │ # => │ │ (#4668) │ │ │ # => 15 │ 4ebbe07d │ Polars upgrade (#4665) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 16 │ 10ceac99 │ menu keybindings in │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ default file (#4651) │ │ l.com │ # => 17 │ 446c2aab │ Lets internals also │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ have exit codes (#4664) │ │ eply.github.com │ # => 18 │ 995757c0 │ flags for find (#4663) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 19 │ 799fa984 │ Update reedline, revert │ Stefan Holderbach │ sholderbach@users.norep │ 3 days ago # => │ │ crossterm (#4657) │ │ ly.github.com │ # => 20 │ d2bd71d2 │ add LAST_EXIT_CODE │ LordMZTE │ lord@mzte.de │ 3 days ago # => │ │ variable (#4655) │ │ │ # => 21 │ 11bc0565 │ Find with regex flag │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4649) │ │ l.com │ # => 22 │ 3eca43c0 │ Plugins without file │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4650) │ │ l.com │ # => 23 │ ed46f0ea │ fix: add missing │ Jae-Heon Ji │ 32578710+jaeheonji@user │ 3 days ago # => │ │ metadata for `ls_colors` │ │ s.noreply.github.com │ # => │ │ (#4603) │ │ │ # => 24 │ 0c3ea636 │ Add support for stderr │ Sophia │ 547158+sophiajt@users.nor │ 4 days ago # => │ │ and exit code (#4647) │ │ eply.github.com │ # => ────┴──────────┴──────────────────────────┴───────────────────┴─────────────────────────┴────────────── # => ... ``` Or even show me all the commits in the last 7 days. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD -n 25 | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 7day)) # => ────┬──────────┬──────────────────────────┬───────────────────┬───────────────────────────┬────────────── # => # │ commit │ subject │ name │ email │ date # => ────┼──────────┼──────────────────────────┼───────────────────┼───────────────────────────┼────────────── # => 0 │ 42f1874a │ Update some examples │ Justin Ma │ hustcer@outlook.com │ 7 hours ago # => │ │ and docs (#4682) │ │ │ # => 1 │ 2a89936b │ Move to latest stable │ Sophia │ 547158+sophiajt@users.nor │ 8 hours ago # => │ │ crossterm, with fix │ │ eply.github.com │ # => │ │ (#4684) │ │ │ # => 2 │ ece5e7db │ dataframe list command │ Fernando Herrera │ fernando.j.herrera@gmai │ 8 hours ago # => │ │ (#4681) │ │ l.com │ # => 3 │ a6a96b29 │ Add binary literals │ Sophia │ 547158+sophiajt@users.nor │ 21 hours ago # => │ │ (#4680) │ │ eply.github.com │ # => 4 │ e3100e6a │ Fix alias in │ Luca Trevisani │ lucatrv@hotmail.com │ a day ago # => │ │ `docs/sample_config/con │ │ │ # => │ │ fig.toml` │ │ │ # => │ │ (#4669) │ │ │ # => 5 │ cb5c61d2 │ Fix open ended ranges │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ (#4677) │ │ eply.github.com │ # => 6 │ b09acdb7 │ Fix unsupported type │ Justin Ma │ hustcer@outlook.com │ a day ago # => │ │ message for some math │ │ │ # => │ │ related commands (#4672) │ │ │ # => 7 │ 0924975b │ Use default_config.nu │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ by default (#4675) │ │ eply.github.com │ # => 8 │ d6a6c4b0 │ Add back in default │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ keybindings (#4673) │ │ eply.github.com │ # => 9 │ eec17304 │ Add profiling build │ Stefan Holderbach │ sholderbach@users.norep │ a day ago # => │ │ profile and symbol strip │ │ ly.github.com │ # => │ │ (#4630) │ │ │ # => 10 │ 10364c4f │ don't use table │ Sophia │ 547158+sophiajt@users.nor │ a day ago # => │ │ compaction in to nuon if │ │ eply.github.com │ # => │ │ not a table (#4671) │ │ │ # => 11 │ ef70c8db │ Date parse refactor │ Jonathan Moore │ jtm170330@utdallas.edu │ 2 days ago # => │ │ (#4661) │ │ │ # => 12 │ 0f437589 │ add last exit code to │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ starship parameters │ │ reply.github.com │ # => │ │ (#4670) │ │ │ # => 13 │ 796d4920 │ add char separators │ Darren Schroeder │ 343840+fdncred@users.no │ 2 days ago # => │ │ (#4667) │ │ reply.github.com │ # => 14 │ 78192100 │ Add shortcircuiting │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ boolean operators │ │ eply.github.com │ # => │ │ (#4668) │ │ │ # => 15 │ 4ebbe07d │ Polars upgrade (#4665) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 16 │ 10ceac99 │ menu keybindings in │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ default file (#4651) │ │ l.com │ # => 17 │ 446c2aab │ Lets internals also │ Sophia │ 547158+sophiajt@users.nor │ 2 days ago # => │ │ have exit codes (#4664) │ │ eply.github.com │ # => 18 │ 995757c0 │ flags for find (#4663) │ Fernando Herrera │ fernando.j.herrera@gmai │ 2 days ago # => │ │ │ │ l.com │ # => 19 │ 799fa984 │ Update reedline, revert │ Stefan Holderbach │ sholderbach@users.norep │ 3 days ago # => │ │ crossterm (#4657) │ │ ly.github.com │ # => 20 │ d2bd71d2 │ add LAST_EXIT_CODE │ LordMZTE │ lord@mzte.de │ 3 days ago # => │ │ variable (#4655) │ │ │ # => 21 │ 11bc0565 │ Find with regex flag │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4649) │ │ l.com │ # => 22 │ 3eca43c0 │ Plugins without file │ Fernando Herrera │ fernando.j.herrera@gmai │ 3 days ago # => │ │ (#4650) │ │ l.com │ # => 23 │ ed46f0ea │ fix: add missing │ Jae-Heon Ji │ 32578710+jaeheonji@user │ 3 days ago # => │ │ metadata for `ls_colors` │ │ s.noreply.github.com │ # => │ │ (#4603) │ │ │ # => 24 │ 0c3ea636 │ Add support for stderr │ Sophia │ 547158+sophiajt@users.nor │ 4 days ago # => │ │ and exit code (#4647) │ │ eply.github.com │ # => ────┴──────────┴──────────────────────────┴───────────────────┴───────────────────────────┴────────────── ``` Now, with the 365 day slice of data, let's `group-by` name where the commits are less than a year old. This table has a lot of columns so it's unreadable. However, if we `group-by` name and `transpose` the table things will look much cleaner. `Pivot` takes rows and turns them into columns or turns columns into rows. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) | group-by name | transpose # => ─────┬─────────────────────────────────┬────────────────── # => # │ column0 │ column1 # => ─────┼─────────────────────────────────┼────────────────── # => 0 │ Justin Ma │ [table 21 rows] # => 1 │ Sophia │ [table 851 rows] # => 2 │ Fernando Herrera │ [table 176 rows] # => 3 │ Luca Trevisani │ [table 1 row] # => 4 │ Stefan Holderbach │ [table 19 rows] # => 5 │ Jonathan Moore │ [table 2 rows] # => 6 │ Darren Schroeder │ [table 242 rows] # => 7 │ LordMZTE │ [table 1 row] # => 8 │ Jae-Heon Ji │ [table 10 rows] # => 9 │ zkldi │ [table 1 row] # => 10 │ Michael Angerman │ [table 61 rows] # => ... ``` Side note: If you happen to get errors, pay attention to the error message. For instance, this error means that the data being returned from `git log` is somehow incomplete. Specifically, there is a missing date column. I've seen git commands work perfectly on Windows and not work at all on Linux or Mac. I'm not sure why. If you run into this issue, one easy way to temporarily avoid it is to limit `git log` results to a certain number like `git log -n 100`. ``` error: Unknown column ┌─ shell:1:124 │ 1 │ git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) │ ^^^^ │ │ │ There isn't a column named 'date' │ Perhaps you meant 'commit'? Columns available: commit, subject ``` Here's one tip for dealing with this error. We have a `do` command that has an `--ignore_errors` parameter. This is how you'd use it in the above example, if it were giving errors. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | do -i { split column "»¦«" commit subject name email date } | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) | group-by name | transpose ``` Now, back to parsing. What if we throw in the `sort-by` and `reverse` commands for good measure? Also, while we're in there, let's get rid of the `[table 21 rows]` thing too. We do that by using the `length` command on each row of column1. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | where ($it.date > ((date now) - 365day)) | group-by name | transpose | upsert column1 {|c| $c.column1 | length} | sort-by column1 | reverse # => ─────┬─────────────────────────────────┬───────── # => # │ column0 │ column1 # => ─────┼─────────────────────────────────┼───────── # => 0 │ Sophia │ 851 # => 1 │ Darren Schroeder │ 242 # => 2 │ Fernando Herrera │ 176 # => 3 │ Jakub Žádník │ 136 # => 4 │ Michael Angerman │ 61 # => 5 │ Andrés N. Robalino │ 29 # => 6 │ Luccas Mateus │ 27 # => 7 │ Stefan Stanciulescu │ 27 # => 8 │ Sophia Turner │ 23 # => 9 │ Tanishq Kancharla │ 21 # => 10 │ Justin Ma │ 21 # => 11 │ onthebridgetonowhere │ 20 # => 12 │ xiuxiu62 │ 19 # => ... ``` This is still a lot of data so let's just look at the top 10 and use the `rename` command to name the columns. We could've also provided the column names with the `transpose` command. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | group-by name | transpose | upsert column1 {|c| $c.column1 | length} | sort-by column1 | rename name commits | reverse | first 10 # => ───┬────────────────────┬───────── # => # │ name │ commits # => ───┼────────────────────┼───────── # => 0 │ Sophia Turner │ 1420 # => 1 │ Sophia │ 851 # => 2 │ Andrés N. Robalino │ 383 # => 3 │ Darren Schroeder │ 380 # => 4 │ Fernando Herrera │ 176 # => 5 │ Yehuda Katz │ 165 # => 6 │ Jakub Žádník │ 140 # => 7 │ Joseph T. Lyons │ 87 # => 8 │ Michael Angerman │ 71 # => 9 │ Jason Gedge │ 67 # => ───┴────────────────────┴───────── ``` And there you have it. The top 10 committers and we learned a little bit of parsing along the way. Here's one last little known command. Perhaps you don't want your table numbered starting with 0. Here's a way to change that with the `table` command. ```nu git log --pretty=%h»¦«%s»¦«%aN»¦«%aE»¦«%aD | lines | split column "»¦«" commit subject name email date | upsert date {|d| $d.date | into datetime} | group-by name | transpose | upsert column1 {|c| $c.column1 | length} | sort-by column1 | rename name commits | reverse | first 10 | table -i 1 # => ────┬────────────────────┬───────── # => # │ name │ commits # => ────┼────────────────────┼───────── # => 1 │ Sophia Turner │ 1420 # => 2 │ Sophia │ 851 # => 3 │ Andrés N. Robalino │ 383 # => 4 │ Darren Schroeder │ 380 # => 5 │ Fernando Herrera │ 176 # => 6 │ Yehuda Katz │ 165 # => 7 │ Jakub Žádník │ 140 # => 8 │ Joseph T. Lyons │ 87 # => 9 │ Michael Angerman │ 71 # => 10 │ Jason Gedge │ 67 ``` Created on 11/9/2020 with Nushell on Windows 10. Updated on 3/1/2022 with Nushell on Windows 10. | key | value | | --- | --- | | version | 0.59.0 | | branch | main | | short\_commit | b09acdb7 | | commit\_hash | b09acdb7f98ec9694cfb223222577bc02ebba4a4 | | commit\_date | 2022-02-28 15:14:33 +00:00 | | build\_os | windows-x86\_64 | | rust\_version | rustc 1.59.0 (9d1b2106e 2022-02-23) | | rust\_channel | stable-x86\_64-pc-windows-msvc | | cargo\_version | cargo 1.59.0 (49d8809dc 2022-02-10) | | pkg\_version | 0.59.0 | | build\_time | 2022-02-28 16:16:00 -06:00 | | build\_rust\_channel | debug | | features | dataframe, default, trash, which, zip | | installed\_plugins | gstat | <|firecrawl-page-2-lllmstxt|> ## HTTP Cookbook Guide # [HTTP](https://www.nushell.sh/cookbook/http.html\#http) ### [Fetching JSON from a url](https://www.nushell.sh/cookbook/http.html\#fetching-json-from-a-url) ```nu http get https://jsonplaceholder.typicode.com/posts | first 5 # => ━━━┯━━━━━━━━┯━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # => # │ userId │ id │ title │ body # => ───┼────────┼────┼─────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────── # => 0 │ 1 │ 1 │ sunt aut facere repellat provident occaecati excepturi │ quia et suscipit # => │ │ │ optio reprehenderit │ suscipit recusandae consequuntur expedita et cum # => │ │ │ │ reprehenderit molestiae ut ut quas totam # => │ │ │ │ nostrum rerum est autem sunt rem eveniet architecto # => 1 │ 1 │ 2 │ qui est esse │ est rerum tempore vitae # => │ │ │ │ sequi sint nihil reprehenderit dolor beatae ea dolores # => │ │ │ │ neque # => │ │ │ │ fugiat blanditiis voluptate porro vel nihil molestiae ut # => │ │ │ │ reiciendis # => │ │ │ │ qui aperiam non debitis possimus qui neque nisi nulla # => 2 │ 1 │ 3 │ ea molestias quasi exercitationem repellat qui ipsa sit │ et iusto sed quo iure # => │ │ │ aut │ voluptatem occaecati omnis eligendi aut ad # => │ │ │ │ voluptatem doloribus vel accusantium quis pariatur # => │ │ │ │ molestiae porro eius odio et labore et velit aut # => 3 │ 1 │ 4 │ eum et est occaecati │ ullam et saepe reiciendis voluptatem adipisci # => │ │ │ │ sit amet autem assumenda provident rerum culpa # => │ │ │ │ quis hic commodi nesciunt rem tenetur doloremque ipsam # => │ │ │ │ iure # => │ │ │ │ quis sunt voluptatem rerum illo velit # => 4 │ 1 │ 5 │ nesciunt quas odio │ repudiandae veniam quaerat sunt sed # => │ │ │ │ alias aut fugiat sit autem sed est # => │ │ │ │ voluptatem omnis possimus esse voluptatibus quis # => │ │ │ │ est aut tenetur dolor neque # => ━━━┷━━━━━━━━┷━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` * * * ### [Fetch from multiple urls](https://www.nushell.sh/cookbook/http.html\#fetch-from-multiple-urls) Suppose you are querying several endpoints, perhaps with different query parameters and you want to view all the responses as a single dataset. An example JSON file, `urls.json`, with the following contents: ```json { "urls": [\ "https://jsonplaceholder.typicode.com/posts/1",\ "https://jsonplaceholder.typicode.com/posts/2",\ "https://jsonplaceholder.typicode.com/posts/3"\ ] } ``` ```nu open urls.json | get urls | each { |u| http get $u } # => ━━━┯━━━━━━━━┯━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # => # │ userId │ id │ title │ body # => ───┼────────┼────┼─────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────── # => 0 │ 1 │ 1 │ sunt aut facere repellat provident occaecati excepturi │ quia et suscipit # => │ │ │ optio reprehenderit │ suscipit recusandae consequuntur expedita et cum # => │ │ │ │ reprehenderit molestiae ut ut quas totam # => │ │ │ │ nostrum rerum est autem sunt rem eveniet architecto # => 1 │ 1 │ 2 │ qui est esse │ est rerum tempore vitae # => │ │ │ │ sequi sint nihil reprehenderit dolor beatae ea dolores # => │ │ │ │ neque # => │ │ │ │ fugiat blanditiis voluptate porro vel nihil molestiae ut # => │ │ │ │ reiciendis # => │ │ │ │ qui aperiam non debitis possimus qui neque nisi nulla # => 2 │ 1 │ 3 │ ea molestias quasi exercitationem repellat qui ipsa sit │ et iusto sed quo iure # => │ │ │ aut │ voluptatem occaecati omnis eligendi aut ad # => │ │ │ │ voluptatem doloribus vel accusantium quis pariatur # => │ │ │ │ molestiae porro eius odio et labore et velit aut # => ━━━┷━━━━━━━━┷━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` * * * If you specify the `--raw` flag, you'll see 3 separate json objects, one in each row. ```nu open urls.json | get urls | each { |u| http get $u -r } # => ━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # => # │ # => ───┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── # => 0 │ { # => │ "userId": 1, # => │ "id": 1, # => │ "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit", # => │ "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum # => │ rerum est autem sunt rem eveniet architecto" # => │ } # => 1 │ { # => │ "userId": 1, # => │ "id": 2, # => │ "title": "qui est esse", # => │ "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro # => │ vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla" # => │ } # => 2 │ { # => │ "userId": 1, # => │ "id": 3, # => │ "title": "ea molestias quasi exercitationem repellat qui ipsa sit aut", # => │ "body": "et iusto sed quo iure\nvoluptatem occaecati omnis eligendi aut ad\nvoluptatem doloribus vel accusantium quis # => │ pariatur\nmolestiae porro eius odio et labore et velit aut" # => │ } # => ━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` * * * To combine these responses together into a valid JSON array, you can turn the table into json. ```nu open urls.json | get urls | each { |u| http get $u } | to json ``` Output ```json [\ {\ "userId": 1,\ "id": 1,\ "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit",\ "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto"\ },\ {\ "userId": 1,\ "id": 2,\ "title": "qui est esse",\ "body": "est rerum tempore vitae\nsequi sint nihil reprehenderit dolor beatae ea dolores neque\nfugiat blanditiis voluptate porro vel nihil molestiae ut reiciendis\nqui aperiam non debitis possimus qui neque nisi nulla"\ },\ {\ "userId": 1,\ "id": 3,\ "title": "ea molestias quasi exercitationem repellat qui ipsa sit aut",\ "body": "et iusto sed quo iure\nvoluptatem occaecati omnis eligendi aut ad\nvoluptatem doloribus vel accusantium quis pariatur\nmolestiae porro eius odio et labore et velit aut"\ }\ ] ``` * * * Making a `post` request to an endpoint with a JSON payload. To make long requests easier, you can organize your json payloads inside a file. ```json { "my_payload": { "title": "foo", "body": "bar", "userId": 1 } } ``` ```nu open payload.json | get my_payload | to json | http post https://jsonplaceholder.typicode.com/posts $in # => ━━━━━ # => id # => ───── # => 101 # => ━━━━━ ``` * * * We can put this all together into a pipeline where we read data, manipulate it, and then send it back to the API. Lets `fetch` a post, `increment` the id, and `post` it back to the endpoint. In this particular example, the test endpoint gives back an arbitrary response which we can't actually mutate. ```nu open urls.json | get urls | first | http get $in | upsert id {|item| $item.id | inc} | to json | http post https://jsonplaceholder.typicode.com/posts $in # => ━━━━━ # => id # => ───── # => 101 # => ━━━━━ ``` ### [Uploading files](https://www.nushell.sh/cookbook/http.html\#uploading-files) To upload a form with a file (think a common file upload form in a browser, where you have to select a file and provide some additional data), you need to: 1. Specify the content type as `multipart/form-data` 2. Provide the record as the POST body 3. Provide the file data in one of the record fields as _binary_ data. ```nu http post https://httpbin.org/post --content-type "multipart/form-data" { icon: (open -r ~/Downloads/favicon-32x32.png), description: "Small icon" } # => ╭─────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────╮ # => │ args │ {record 0 fields} │ # => │ data │ │ # => │ │ ╭──────┬────────────────────────────────────────────────────────────────────────────────────────────╮ │ # => │ files │ │ icon │ data:application/octet-stream;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAACXBIW │ │ # => │ │ │ │ XMAAAsTAAALEwEAmpwYAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAG5SURBVHgBrZeBUcMwDEU/XYBuUG8 │ │ # => │ │ │ │ AG2A26AZ0A7pBu0FhgmaDskHKBA0TJExAmSBYd/bFBNmWfLw73fUukvXlWI4KpLHOTs56Z6OzL2ets03C3zg7MP47/ │ │ # => │ │ │ │ 0zM0geOGeuZRW3BfwsBBlMFJaMK74UCghVFHIXJ48qWCgHjTPSf6scK2ysFtHHSRfRb9I4YHqDDYtq1XwLuUIeFHgt │ │ # => │ │ │ │ GgEE9K+hgd+CKer6h48oJ+EAdA/TiBzACGtRxho7BWZd6SC2iaUG6jIyPtcKYDTIYv6hUQNy6VuD/AgF0U/UoVz6/N │ │ # => │ │ │ │ 2whpoEC4wN6JnELvmVNQniLzF1xgzK0I9S3dNIHlE988If3H3LOC5QJCZeQMUQx1XcLJduBP5BHpF9BC/4VbKBAcgj │ │ # => │ │ │ │ nHUDYgv8BAgx0bfikECASIal83hXagWQdJ4wP4Rr6LyIl184Rz6kHR+iqD9b7eKuIWYWk8Q4kZ7UCBvIWDTxyArSLx │ │ # => │ │ │ │ Nyikv8aSD6hgx1I3lFHBz0dJ+ANdbxCxxmZ7wP9F6zpAMIKY7KHnQ7iRbhQPA1JBewhgEQ0KFduZnG2IFb9x4duxhO │ │ # => │ │ │ │ mb0MYRrYF4ZeZ0D0yN+wPKKVmaKtbyvUAAAAASUVORK5CYII= │ │ # => │ │ ╰──────┴────────────────────────────────────────────────────────────────────────────────────────────╯ │ # => │ │ ╭─────────────┬────────────╮ │ # => │ form │ │ description │ Small icon │ │ # => │ │ ╰─────────────┴────────────╯ │ # => │ │ ╭────────────────────────┬──────────────────────────────────────────────────────────────────────────╮ │ # => │ headers │ │ Accept │ */* │ │ # => │ │ │ Accept-Encoding │ gzip │ │ # => │ │ │ Content-Length │ 893 │ │ # => │ │ │ Content-Type │ multipart/form-data; boundary=cddfac9d-e5e0-4aa3-a3df-6f9f6e570bc9 │ │ # => │ │ │ Host │ httpbin.org │ │ # => │ │ │ User-Agent │ nushell │ │ # => │ │ │ X-Amzn-Trace-Id │ Root=1-66b28d98-549045021ddb79ab3d0eda79 │ │ # => │ │ ╰────────────────────────┴──────────────────────────────────────────────────────────────────────────╯ │ # => │ json │ │ # => │ url │ https://httpbin.org/post │ # => ╰─────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` If the file happens to be a text file, you may need to additionally convert it to binary data before sending it. This can be done using the `into binary` command. ```nu http post https://httpbin.org/post --content-type "multipart/form-data" { doc: (open -r ~/Downloads/README.txt | into binary), description: "Documentation file" } # => ╭─────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────╮ # => │ args │ {record 0 fields} │ # => │ data │ │ # => │ │ ╭──────┬───────────────────────────────────────────────────────────────────────────────────────────────╮ │ # => │ files │ │ doc │ To use Nu plugins, use the register command to tell Nu where to find the plugin. For example: │ │ # => │ │ │ │ │ │ # => │ │ │ │ > register ./nu_plugin_query │ │ # => │ │ ╰──────┴───────────────────────────────────────────────────────────────────────────────────────────────╯ │ # => │ │ ╭─────────────┬────────────────────╮ │ # => │ form │ │ description │ Documentation file │ │ # => │ │ ╰─────────────┴────────────────────╯ │ # => │ │ ╭─────────────────┬────────────────────────────────────────────────────────────────────╮ │ # => │ headers │ │ Accept │ */* │ │ # => │ │ │ Accept-Encoding │ gzip │ │ # => │ │ │ Content-Length │ 476 │ │ # => │ │ │ Content-Type │ multipart/form-data; boundary=f872d6c3-7937-426d-b266-de562b777e1d │ │ # => │ │ │ Host │ httpbin.org │ │ # => │ │ │ User-Agent │ nushell │ │ # => │ │ │ X-Amzn-Trace-Id │ Root=1-66b28eef-4998c6ab0ef5becb19ca7f6f │ │ # => │ │ ╰─────────────────┴────────────────────────────────────────────────────────────────────╯ │ # => │ json │ │ # => │ url │ https://httpbin.org/post │ # => ╰─────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` <|firecrawl-page-3-lllmstxt|> ## Pattern Matching in Nu # [Pattern Matching](https://www.nushell.sh/cookbook/pattern_matching.html\#pattern-matching) ## [Using the `match` keyword](https://www.nushell.sh/cookbook/pattern_matching.html\#using-the-match-keyword) Like many other languages, nu offers a [`match`](https://www.nushell.sh/commands/docs/match.html#frontmatter-title-for-core) keyword. Usually this is used as a slightly more ergonomic version of `if-else` statements if you have many branches ```nu [black red yellow green purple blue indigo] | each {|c| match $c { "black" => "classy" "red" | "green" | "blue" => "fundamental" "yellow" | "purple" => "vibrant" _ => "innovative" } } # => ───┬──────────── # => 0 │ classy # => 1 │ funamental # => 2 │ vibrant # => 3 │ funamental # => 4 │ vibrant # => 5 │ funamental # => 6 │ innovative # => ───┴──────────── ``` The equivalent in `if-else` statements would be: ```nu [black red yellow green purple blue] | each {|c| if ($c == "black") { "classy" } else if ($c in ["red", "green", "blue"]) { "fundamental" } else if ($c in ['yellow', "purple"]) { "vibrant" } else { "innovative" } } ``` As you can see you can also use command expressions in match statements (in this case used with `|`). Also notice the `_` case at the end, this is called the default arm and is used in case none of the other patterns match. Note also that in the case that cases overlap the first matching pattern will be used (just like with `if-else` statements): ```nu [yellow green] | each {|c| match $c { "green" => "fundamental" "yellow" | "green" => "vibrant" } } # => ───┬──────────── # => 0 │ vibrant # => 1 │ funamental # => ───┴──────────── ``` ## [Pattern matching on types](https://www.nushell.sh/cookbook/pattern_matching.html\#pattern-matching-on-types) You can use the [`describe`](https://www.nushell.sh/commands/docs/describe.html) command to get more info about the types of values. For example: ```nu {one: 1 two: 2} | describe # => record ``` ```nu [{a: 1 b: 2} {a: 2 b:3 }] | describe # => table ``` Together with `match` and some clever regex use you can do quite powerful type matching. For example, let's say we wanted to implement a `str append` function that would work on both strings and lists. On strings it would work as expected, on lists of strings, it should append the same string to each element of the list. Using `match` one might do that like so: ```nu def "str append" [tail: string]: [string -> string, list -> list] { let input = $in match ($input | describe | str replace --regex '<.*' '') { "string" => { $input ++ $tail }, "list" => { $input | each {|el| $el ++ $tail} }, _ => $input } } ``` The `$input | describe` would output for example `string` if the input was a string, and for example `list` for a list containing multiple different types. The regex, removes everying after the first `<` leaving us just with `list`. Then with the `match` statement we can handle the different types separately. Finally in the default case we just return the input unaltered so that other types can simply pass through this filter without issue. Also note that we have to capture the `$in` variable on the first statement of the function to still have access to it in each `match` arm. With this implementation we can check that the command works as expected: ```nu use std/assert assert equal ("foo" | str append "/") "foo/" assert equal (["foo", "bar", "baz"] | str append "/") ["foo/", "bar/", "baz/"] ``` <|firecrawl-page-4-lllmstxt|> ## SSH Agent Management # [Manage SSH passphrases](https://www.nushell.sh/cookbook/ssh_agent.html\#manage-ssh-passphrases) `eval` is not available in nushell, so run: ```nu ^ssh-agent -c | lines | first 2 | parse "setenv {name} {value};" | transpose -r | into record | load-env ``` Warning Adding this to your `env.nu` will however start a new ssh-agent process every time you start a new terminal. See the workarounds. Alternatively, use the third-party Nu plugin [bash-env](https://github.com/tesujimath/nu_plugin_bash_env) as follows. ```nu ^ssh-agent | bash-env | load-env ``` Warning Please note that the `bash-env` plugin is not supported by the core Nushell team. All issues and requests for support should be directed to its own [issue tracker](https://github.com/tesujimath/nu_plugin_bash_env/issues). ## [Workarounds](https://www.nushell.sh/cookbook/ssh_agent.html\#workarounds) You can work around this behavior by checking if a ssh-agent is already running on your user, and start one if none is: ```nu do --env { let ssh_agent_file = ( $nu.temp-path | path join $"ssh-agent-($env.USER? | default $env.USERNAME).nuon" ) if ($ssh_agent_file | path exists) { let ssh_agent_env = open ($ssh_agent_file) if ($"/proc/($ssh_agent_env.SSH_AGENT_PID)" | path exists) { load-env $ssh_agent_env return } else { rm $ssh_agent_file } } let ssh_agent_env = ^ssh-agent -c | lines | first 2 | parse "setenv {name} {value};" | transpose --header-row | into record load-env $ssh_agent_env $ssh_agent_env | save --force $ssh_agent_file } ``` ### [Keychain](https://www.funtoo.org/Funtoo:Keychain) ```nu keychain --eval --quiet | lines | where not ($it | is-empty) | parse "{k}={v}; export {k2};" | select k v | transpose --header-row | into record | load-env ``` ## [Non-nushell workarounds](https://www.nushell.sh/cookbook/ssh_agent.html\#non-nushell-workarounds) However, the commonly recommended approach involves running an ssh-agent so it establishes an user-wide socket for processes to connect to. Here are two common ways to achieve this. ### [DE/WM config](https://www.nushell.sh/cookbook/ssh_agent.html\#de-wm-config) You can incorporate it into your Desktop Environment (DE) or Compositor's configuration using the following command: ```sh ssh-agent -D -a /run/user/1000/ssh-agent.socket # You can also set this socket path as an environment variable using the same config file ``` This a good option for you if you're using a Windows Manager or a Compositor since you're likely to know its syntax. ### [As a service](https://www.nushell.sh/cookbook/ssh_agent.html\#as-a-service) Alternatively, you can enable it as an **user service**. OpenSSH typically includes a systemd service and the [ArchLinux wiki systemd/User](https://wiki.archlinux.org/title/Systemd/User) page covers how to enable services per user with systemd. However, if you're using a different service manager, please refer its own documentation to create a user service that utilizes the aforementioned command. To enable Nushell to access this socket, you need to add its path as `$env.SSH_AUTH_SOCK` like so: ```nu $env.SSH_AUTH_SOCK = $"($env.XDG_RUNTIME_DIR)/ssh-agent.socket" ``` <|firecrawl-page-5-lllmstxt|> ## External Completers Guide # [External Completers](https://www.nushell.sh/cookbook/external_completers.html\#external-completers) ## [Completers](https://www.nushell.sh/cookbook/external_completers.html\#completers) ### [Carapace completer](https://www.nushell.sh/cookbook/external_completers.html\#carapace-completer) ```nu let carapace_completer = {|spans| carapace $spans.0 nushell ...$spans | from json } ``` ### [Fish completer](https://www.nushell.sh/cookbook/external_completers.html\#fish-completer) This completer will use [the fish shell](https://fishshell.com/) to handle completions. Fish handles out of the box completions for many popular tools and commands. ```nu let fish_completer = {|spans| fish --command $"complete '--do-complete=($spans | str replace --all "'" "\\'" | str join ' ')'" | from tsv --flexible --noheaders --no-infer | rename value description | update value {|row| let value = $row.value let need_quote = ['\' ',' '[' ']' '(' ')' ' ' '\t' "'" '"' "`"] | any {$in in $value} if ($need_quote and ($value | path exists)) { let expanded_path = if ($value starts-with ~) {$value | path expand --no-symlink} else {$value} $'"($expanded_path | str replace --all "\"" "\\\"")"' } else {$value} } } ``` A couple of things to note on this command: - The fish completer will return lines of text, each one holding the `value` and `description` separated by a tab. The `description` can be missing, and in that case there won't be a tab after the `value`. If that happens, `from tsv` will fail, so we add the `--flexible` flag. - The output of the fish completer does not contain a header (name of the columns), so we add `--noheaders` to prevent `from tsv` from treating the first row as headers and later give the columns their names using `rename`. - `--no-infer` is optional. `from tsv` will infer the data type of the result, so a numeric value like some git hashes will be inferred as a number. `--no-infer` will keep everything as a string. It doesn't make a difference in practice but it will print a more consistent output if the completer is ran on it's own. - Since fish only supports POSIX style escapes for file paths ( `file\ name.txt`, etc.), file paths completed by fish will not be quoted or escaped properly on external commands. Nushell does not parse POSIX escapes, so we need to do this conversion manually such as by testing if the items are valid paths as shown in the example. To minimize the overhead of path lookups, we first check the string for common escape characters. If the string needs escaping, and it is a path on the filesystem, then the value is double-quoted. Also before double-quoting the file path we expand any ~ at the beginning of the path, so that completions continue to work. This simple approach is imperfect, but it should cover 99.9% of use cases. ### [Multiple completer](https://www.nushell.sh/cookbook/external_completers.html\#multiple-completer) Sometimes, a single external completer is not flexible enough. Luckily, as many as needed can be combined into a single one. The following example uses `$default_completer` for all commands except the ones explicitly defined in the record: ```nu let multiple_completers = {|spans| match $spans.0 { ls => $ls_completer git => $git_completer _ => $default_completer } | do $in $spans } ``` > **Note** In the example above, `$spans.0` is the command being run at the time. The completer will match the desired completer, and fallback to `$default_completer`. > > - If we try to autocomplete `git `, `spans` will be `[git ""]`. `match $spans.0 { ... }` will return the `$git_completer`. > - If we try to autocomplete `other_command `, `spans` will be `[other_command ""]`. The match will fallback to the default case ( `_`) and return the `$default_completer`. ## [Troubleshooting](https://www.nushell.sh/cookbook/external_completers.html\#troubleshooting) ### [Alias completions](https://www.nushell.sh/cookbook/external_completers.html\#alias-completions) Nushell currently has a [bug where autocompletions won't work for aliases](https://github.com/nushell/nushell/issues/8483). This can be worked around adding the following snippet at the beginning of the completer: ```nu # if the current command is an alias, get it's expansion let expanded_alias = (scope aliases | where name == $spans.0 | get -i 0 | get -i expansion) # overwrite let spans = (if $expanded_alias != null { # put the first word of the expanded alias first in the span $spans | skip 1 | prepend ($expanded_alias | split row " " | take 1) } else { $spans }) ``` This code will take the first span, find the first alias that matches it, and replace the beginning of the command with the alias expansion. ### [`ERR unknown shorthand flag` using carapace](https://www.nushell.sh/cookbook/external_completers.html\#err-unknown-shorthand-flag-using-carapace) Carapace will return this error when a non-supported flag is provided. For example, with `cargo -1`: | value | description | | --- | --- | | -1ERR | unknown shorthand flag: "1" in -1 | | -1\_ | | The solution to this involves manually checking the value to filter it out: ```nu let carapace_completer = {|spans: list| carapace $spans.0 nushell ...$spans | from json | if ($in | default [] | where value == $"($spans | last)ERR" | is-empty) { $in } else { null } } ``` ## [Putting it all together](https://www.nushell.sh/cookbook/external_completers.html\#putting-it-all-together) This is an example of how an external completer definition might look like: ```nu let fish_completer = ... let carapace_completer = {|spans: list| carapace $spans.0 nushell ...$spans | from json | if ($in | default [] | where value =~ '^-.*ERR$' | is-empty) { $in } else { null } } # This completer will use carapace by default let external_completer = {|spans| let expanded_alias = scope aliases | where name == $spans.0 | get -o 0.expansion let spans = if $expanded_alias != null { $spans | skip 1 | prepend ($expanded_alias | split row ' ' | take 1) } else { $spans } match $spans.0 { # carapace completions are incorrect for nu nu => $fish_completer # fish completes commits and branch names in a nicer way git => $fish_completer # carapace doesn't have completions for asdf asdf => $fish_completer _ => $carapace_completer } | do $in $spans } $env.config = { # ... completions: { external: { enable: true completer: $external_completer } } # ... } ``` <|firecrawl-page-6-lllmstxt|> ## Git Commands Cookbook # [Git](https://www.nushell.sh/cookbook/git.html\#git) Nu can help with common `Git` tasks like removing all local branches which have been merged into master. ### [Delete git merged branches](https://www.nushell.sh/cookbook/git.html\#delete-git-merged-branches) **Warning**: This command will hard delete the merged branches from your machine. You may want to check the branches selected for deletion by omitting the last git command. ```nu git branch --merged | lines | where ($it != "* master" and $it != "* main") | each {|br| git branch -D ($br | str trim) } | str trim # => ───┬─────────────────────────────────────────── # => 0 │ Deleted branch start_urls (was fc01bb45). # => ───┴─────────────────────────────────────────── ``` ### [Parse formatted commit messages (more details in the parsing git log section)](https://www.nushell.sh/cookbook/git.html\#parse-formatted-commit-messages-more-details-in-the-parsing-git-log-section) ```nu git log --pretty=%h»¦«%aN»¦«%s»¦«%aD | lines | split column "»¦«" sha1 committer desc merged_at | first 10 # => ───┬──────────┬───────────────────┬───────────────────────────────────────────────────────┬───────────────────────────────── # => # │ sha1 │ committer │ desc │ merged_at # => ───┼──────────┼───────────────────┼───────────────────────────────────────────────────────┼───────────────────────────────── # => 0 │ 42f1874a │ Justin Ma │ Update some examples and docs (#4682) │ Tue, 1 Mar 2022 21:05:29 +0800 # => 1 │ 2a89936b │ Sophia │ Move to latest stable crossterm, with fix (#4684) │ Tue, 1 Mar 2022 07:05:46 -0500 # => 2 │ ece5e7db │ Fernando Herrera │ dataframe list command (#4681) │ Tue, 1 Mar 2022 11:41:13 +0000 # => 3 │ a6a96b29 │ Sophia │ Add binary literals (#4680) │ Mon, 28 Feb 2022 18:31:53 -0500 # => 4 │ e3100e6a │ Luca Trevisani │ Fix alias in `docs/sample_config/config.toml` (#4669) │ Mon, 28 Feb 2022 22:47:14 +0100 # => 5 │ cb5c61d2 │ Sophia │ Fix open ended ranges (#4677) │ Mon, 28 Feb 2022 11:15:31 -0500 # => 6 │ b09acdb7 │ Justin Ma │ Fix unsupported type message for some math related │ Mon, 28 Feb 2022 23:14:33 +0800 # => │ │ │ commands (#4672) │ # => 7 │ 0924975b │ Sophia │ Use default_config.nu by default (#4675) │ Mon, 28 Feb 2022 10:12:08 -0500 # => 8 │ d6a6c4b0 │ Sophia │ Add back in default keybindings (#4673) │ Mon, 28 Feb 2022 08:54:40 -0500 # => 9 │ eec17304 │ Stefan Holderbach │ Add profiling build profile and symbol strip (#4630) │ Mon, 28 Feb 2022 13:13:24 +0100 # => ───┴──────────┴───────────────────┴───────────────────────────────────────────────────────┴───────────────────────────────── ``` * * * ### [View git committer activity as a `histogram`](https://www.nushell.sh/cookbook/git.html\#view-git-committer-activity-as-a-histogram) _Note: the `histogram` command is not yet ported to the latest version_ ```nu git log --pretty=%h»¦«%aN»¦«%s»¦«%aD | lines | split column "»¦«" sha1 committer desc merged_at | histogram committer merger | sort-by merger | reverse # => ━━━━┯━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ # => # │ committer │ merger # => ────┼─────────────────────┼────────────────────────────────────────────────────────────────────────────────────────────────────── # => 0 │ Sophia Turner │ **************************************************************************************************** # => 1 │ Andrés N. Robalino │ *********************** # => 2 │ Yehuda Katz │ ************** # => 3 │ est31 │ ***** # => 4 │ Thomas Hartmann │ **** # => 5 │ Sean Hellum │ ** # => 6 │ Patrick Meredith │ ** # => 7 │ Fahmi Akbar Wildana │ ** # => 8 │ Vanessa Sochat │ * # => 9 │ Shaurya Shubham │ * # => 10 │ Pirmin Kalberer │ * # => 11 │ Odin Dutton │ * # => 12 │ Jonathan Rothberg │ * # => ━━━┷━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` <|firecrawl-page-7-lllmstxt|> ## NuShell Environment Setup # [Setup](https://www.nushell.sh/cookbook/setup.html\#setup) To get the most out of nu, it is important to setup your path and env for easy access. There are other ways to view these values and variables, however setting up your nu configuration will make it much easier as these have cross-platform support. * * * ### [Configure your path and other environment variables](https://www.nushell.sh/cookbook/setup.html\#configure-your-path-and-other-environment-variables) In your `env.nu`, you can set up your environment. To configure environment variables, you use the `$env` variable: ```nu $env.TITLE = 'Nu Test' $env.VALUE = 123 ``` To add paths to the `PATH` environment variable, you can append them: ```nu $env.PATH ++= ['~/.local/bin'] ``` Because you can append a list of paths, you can append multiple at once. You can also use subcommands to construct the paths in line. ```nu $env.PATH ++= [ '~/.local/bin', ($env.CARGO_HOME | path join "bin") ] ``` Because PATH order makes a difference, you may want to _prepend_ your paths instead, so that they take precedence over other executables with the same name: ``` use std/util "path add" path add '~/.local/bin' ``` For more information, see the documentation about [environment variables](https://www.nushell.sh/book/environment.html#setting-environment-variables) and [PATH configuration](https://www.nushell.sh/book/configuration.html#path-configuration). ### [How to list your environment variables](https://www.nushell.sh/cookbook/setup.html\#how-to-list-your-environment-variables) ```nu $env # => ─────────────────────────────────┬──────────────────────────────────────────── # => ALLUSERSPROFILE │ C:\ProgramData # => CARGO_PKG_AUTHORS │ The Nu Project Contributors # => CARGO_PKG_DESCRIPTION │ A new type of shell # => CARGO_PKG_HOMEPAGE │ https://www.nushell.sh # => CARGO_PKG_LICENSE │ MIT # => CARGO_PKG_LICENSE_FILE │ # => CARGO_PKG_NAME │ nu # => CARGO_PKG_REPOSITORY │ https://github.com/nushell/nushell # => CARGO_PKG_VERSION │ 0.59.0 # => CARGO_PKG_VERSION_MAJOR │ 0 ``` Let's practise that and set `$EDITOR` in our `env.nu` file using `vim` (or an editor of your choice) ```nu vim $nu.env-path ``` Note: if you've never used `vim` before and you want to leave typing `:q!` will close without saving. Go to the end of the file and add ```nu $env.EDITOR = 'vim' ``` or `emacs`, `vscode` or whatever editor you like. Don't forget that the program needs to be accessible on the `PATH` and to reload your configuration with `exec nu` on linux/mac or restart your nushell on windows. You should now be able to run `config nu` or `config env` and edit those files easily. * * * ### [How to get a single environment variable's value](https://www.nushell.sh/cookbook/setup.html\#how-to-get-a-single-environment-variable-s-value) ```nu $env.APPDATA ``` * * * ### [Use hooks to export state via environment variables](https://www.nushell.sh/cookbook/setup.html\#use-hooks-to-export-state-via-environment-variables) Additional tools like starship run with every prompt showing up in nushell. [`starship`](https://starship.rs/) in particular replaces the default prompt with its own. To be most compatible, the `starship` binary will run every prompt render and is absolute stateless. Nushell, however, is very stateful in a single instance. [Hooks](https://www.nushell.sh/book/hooks.html#hooks) allow registration of custom callback functions. In this case, the `pre_prompt` hook is very useful. With it, we can export state information as an environment variable, for example, what [overlays](https://www.nushell.sh/book/overlays.html) are currently activated. ```nu # set NU_OVERLAYS with overlay list, useful for starship prompt $env.config.hooks.pre_prompt = ($env.config.hooks.pre_prompt | append {|| let overlays = overlay list | range 1.. if not ($overlays | is-empty) { $env.NU_OVERLAYS = $overlays | str join ", " } else { $env.NU_OVERLAYS = null } }) ``` Now in `starship`, we can use this environment variable to display what modules are active. ```toml [env_var.NU_OVERLAYS] symbol = '📌 ' format = 'with [$symbol($env_value )]($style)' style = 'red' ``` <|firecrawl-page-8-lllmstxt|> ## Nushell Module Cookbook # [Module Scenarios](https://www.nushell.sh/cookbook/modules.html\#module-scenarios) ## [Dumping Files into Directory](https://www.nushell.sh/cookbook/modules.html\#dumping-files-into-directory) A common pattern in traditional shells is dumping and auto-sourcing files from a directory (for example, loading custom completions). In Nushell, doing this directly is currently not possible, but directory modules can still be used. Here we'll create a simple completion module with a submodule dedicated to some Git completions: 1. Create the completion directory `mkdir ($nu.default-config-dir | path join completions)` 2. Create an empty `mod.nu` for it `touch ($nu.default-config-dir | path join completions mod.nu)` 3. Put the following snippet in `git.nu` under the `completions` directory ```nu export extern main [\ --version(-v)\ -C: string\ # ... etc.\ ] export extern add [\ --verbose(-v)\ --dry-run(-n)\ # ... etc.\ ] export extern checkout [\ branch: string@complete-git-branch\ ] def complete-git-branch [] { # ... code to list git branches } ``` 4. Add `export module git.nu` to `mod.nu` 5. Add the parent of the `completions` directory to your `NU_LIB_DIRS` inside `env.nu` ```nu $env.NU_LIB_DIRS = [\ ...\ $nu.default-config-dir\ ] ``` 6. Import the completions to Nushell in your `config.nu`: `use completions *` Now you've set up a directory where you can put your completion files, and you should have some Git completions the next time you start Nushell. Note This will use the file name (in our example `git` from `git.nu`) as the module name. This means some completions might not work if the definition has the base command in its name. For example, if you defined our known externals in our `git.nu` as `export extern 'git push' []`, etc. and followed the rest of the steps, you would get subcommands like `git git push`, etc. You would need to call `use completions git *` to get the desired subcommands. For this reason, using `main` as outlined in the step above is the preferred way to define subcommands. ## [Overlay and "Virtual Environments"](https://www.nushell.sh/cookbook/modules.html\#overlay-and-virtual-environments) [Overlays](https://www.nushell.sh/book/overlays.html) are layers of definitions. We can make use of them to establish a temporary virtual environment, with custom environment variables, which we discard at the end. Our goals in this example are: - Activate a set of environment variables from a file called `env.json` - Work in this context - Discard the environment - restoring the original environment First, let's prepare an `env.json` for testing: ```nu { A: 1 B: 2 } | save env.json $env.A = 'zzz' print $"('A' in $env) ('B' in $env)" # => true false ``` Now let's create a module `env` with a `load` command that loads the environment from `env.json`, and use it as an overlay: ```nu 'export def --env load [] { open env.json | load-env }' | save env.nu overlay use ./env.nu overlay list # => ╭───┬──────╮ # => │ 0 │ zero │ # => │ 1 │ env │ # => ╰───┴──────╯ ``` Now we load the `env.json` file: ```nu load print $"($env.A) ($env.B)" # => 1 2 ``` To hide the overlay: ```nu overlay hide env print $"('A' in $env) ('B' in $env)" # => true false ``` Note that - as documented in [Overlays](https://www.nushell.sh/book/overlays.html) \- reactivating the overlay will recover the loaded environment variables, not create a new context for as long as the Nushell session remains active, despite `overlay list` no longer listing the overlay. More related information and specifically about environment variables and their modification can be found in [Environment](https://www.nushell.sh/book/environment.html), [Modules](https://www.nushell.sh/book/modules.html), [Overlay](https://www.nushell.sh/book/overlays.html), and the respective command documentation of [`def --env`](https://www.nushell.sh/commands/docs/def.html), [`export def --env`](https://www.nushell.sh/commands/docs/export_def.html), [`load-env`](https://www.nushell.sh/commands/docs/load-env.html), and [`export-env`](https://www.nushell.sh/commands/docs/export-env.html). ### [Elaborate Virtual Environments](https://www.nushell.sh/cookbook/modules.html\#elaborate-virtual-environments) This kind of overlaying environments can be used to scope more elaborate virtual environments, including changing the `PATH` environment variable, or other tool settings defined in environment variables or files. Tools like conda or Python virtualenv manage and isolate sets of environment variables. The [official virtualenv integration](https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu) makes use of these concepts. And our nu\_scripts repository has a an [unofficial Conda module](https://github.com/nushell/nu_scripts/tree/main/modules/virtual_environments). <|firecrawl-page-9-lllmstxt|> ## System Commands Overview # [System](https://www.nushell.sh/cookbook/system.html\#system) Nu offers many commands that help interface with the filesystem and control your operating system. ### [View all files in the current directory](https://www.nushell.sh/cookbook/system.html\#view-all-files-in-the-current-directory) ```nu ls | where type == file # => ────┬─────────────────────────────────┬──────┬──────────┬──────────────── # => # │ name │ type │ size │ modified # => ────┼─────────────────────────────────┼──────┼──────────┼──────────────── # => 0 │ CODE_OF_CONDUCT.md │ file │ 3.5 KB │ 10 months ago # => 1 │ CONTRIBUTING.md │ file │ 1.8 KB │ 10 months ago # => 2 │ Cargo.lock │ file │ 118.4 KB │ 2 hours ago # => 3 │ Cargo.toml │ file │ 4.1 KB │ 2 hours ago # => 4 │ Cargo.toml.old │ file │ 7.2 KB │ 2 weeks ago # => 5 │ LICENSE │ file │ 1.1 KB │ 4 months ago # => 6 │ Makefile.toml │ file │ 473 B │ 10 months ago # => 7 │ README.build.txt │ file │ 193 B │ 10 months ago # => 8 │ README.md │ file │ 15.8 KB │ 3 days ago # => 9 │ bands.txt │ file │ 156 B │ 2 hours ago # => 10 │ extra_features_cargo_install.sh │ file │ 54 B │ 4 months ago # => 11 │ files │ file │ 3 B │ an hour ago # => 12 │ payload.json │ file │ 88 B │ 21 minutes ago # => 13 │ rustfmt.toml │ file │ 16 B │ 10 months ago # => 14 │ urls.json │ file │ 182 B │ 25 minutes ago # => ────┴─────────────────────────────────┴──────┴──────────┴──────────────── ``` * * * ### [View all directories in the current directory](https://www.nushell.sh/cookbook/system.html\#view-all-directories-in-the-current-directory) ```nu ls | where type == dir # => ────┬───────────┬──────┬─────────┬─────────────── # => # │ name │ type │ size │ modified # => ────┼───────────┼──────┼─────────┼─────────────── # => 0 │ .azureold │ dir │ 0 B │ 3 weeks ago # => 1 │ .cargo │ dir │ 0 B │ 10 months ago # => 2 │ .vscode │ dir │ 0 B │ 10 months ago # => 3 │ crates │ dir │ 12.3 KB │ 3 weeks ago # => 4 │ docs │ dir │ 4.1 KB │ a day ago # => 5 │ images │ dir │ 4.1 KB │ 2 weeks ago # => 6 │ pkg_mgrs │ dir │ 0 B │ 10 months ago # => 7 │ samples │ dir │ 0 B │ 10 months ago # => 8 │ src │ dir │ 4.1 KB │ 3 hours ago # => 9 │ target │ dir │ 0 B │ 2 weeks ago # => 10 │ tests │ dir │ 0 B │ 4 months ago # => 11 │ wix │ dir │ 0 B │ 2 weeks ago # => ────┴───────────┴──────┴─────────┴─────────────── ``` * * * ### [Find processes sorted by greatest cpu utilization.](https://www.nushell.sh/cookbook/system.html\#find-processes-sorted-by-greatest-cpu-utilization) ```nu ps | where cpu > 0 | sort-by cpu | reverse # => ───┬───────┬────────────────────┬───────┬─────────┬───────── # => # │ pid │ name │ cpu │ mem │ virtual # => ───┼───────┼────────────────────┼───────┼─────────┼───────── # => 0 │ 11928 │ nu.exe │ 32.12 │ 47.7 MB │ 20.9 MB # => 1 │ 11728 │ Teams.exe │ 10.71 │ 53.8 MB │ 50.8 MB # => 2 │ 21460 │ msedgewebview2.exe │ 8.43 │ 54.0 MB │ 36.8 MB # => ───┴───────┴────────────────────┴───────┴─────────┴───────── ``` * * * ### [Find and kill a hanging process](https://www.nushell.sh/cookbook/system.html\#find-and-kill-a-hanging-process) Sometimes a process doesn't shut down correctly. Using `ps` it's fairly easy to find the pid of this process: ```nu ps | where name == Notepad2.exe # => ───┬──────┬──────────────┬──────┬─────────┬───────── # => # │ pid │ name │ cpu │ mem │ virtual # => ───┼──────┼──────────────┼──────┼─────────┼───────── # => 0 │ 9268 │ Notepad2.exe │ 0.00 │ 32.0 MB │ 9.8 MB # => ───┴──────┴──────────────┴──────┴─────────┴───────── ``` This process can be sent the kill signal in a one-liner: ```nu ps | where name == Notepad2.exe | get pid.0 | kill $in # => ───┬──────────────────────────────────────────────────────────────── # => 0 │ SUCCESS: Sent termination signal to the process with PID 9268. # => ───┴──────────────────────────────────────────────────────────────── ``` Notes: - `kill` is a built-in Nu command that works on all platforms. If you wish to use the classic Unix `kill` command, you can do so with `^kill`. - Filtering with the `where` command as shown above is case-sensitive. <|firecrawl-page-10-lllmstxt|> ## Nu Parsing Techniques # [Parsing](https://www.nushell.sh/cookbook/parsing.html\#parsing) _Nu_ offers the ability to do some basic parsing, with different ways to achieve the same goal. Builtin-functions that can be used include: - `lines` - `detect columns` - `parse` - `str ...` - `from ssv` A few illustrative examples follow. ## [Examples (tabular output)](https://www.nushell.sh/cookbook/parsing.html\#examples-tabular-output) ### [`detect columns` (pretty automatic)](https://www.nushell.sh/cookbook/parsing.html\#detect-columns-pretty-automatic) ```nu df -h | str replace "Mounted on" Mounted_On | detect columns # => ╭────┬───────────────────────────────────┬──────┬──────┬───────┬──────┬────────────────────────────────────╮ # => │ # │ Filesystem │ Size │ Used │ Avail │ Use% │ Mounted_On │ # => ├────┼───────────────────────────────────┼──────┼──────┼───────┼──────┼────────────────────────────────────┤ # => │ 0 │ devtmpfs │ 3.2G │ 0 │ 3.2G │ 0% │ /dev │ # => │ 1 │ tmpfs │ 32G │ 304M │ 32G │ 1% │ /dev/shm │ # => │ 2 │ tmpfs │ 16G │ 11M │ 16G │ 1% │ /run │ # => │ 3 │ tmpfs │ 32G │ 1.2M │ 32G │ 1% │ /run/wrappers │ # => │ 4 │ /dev/nvme0n1p2 │ 129G │ 101G │ 22G │ 83% │ / │ # => │ 5 │ /dev/nvme0n1p8 │ 48G │ 16G │ 30G │ 35% │ /var │ # => │ 6 │ efivarfs │ 128K │ 24K │ 100K │ 20% │ /sys/firmware/efi/efivars │ # => │ 7 │ tmpfs │ 32G │ 41M │ 32G │ 1% │ /tmp │ # => │ 9 │ /dev/nvme0n1p3 │ 315G │ 230G │ 69G │ 77% │ /home │ # => │ 10 │ /dev/nvme0n1p1 │ 197M │ 120M │ 78M │ 61% │ /boot │ # => │ 11 │ /dev/mapper/vgBigData-lvBigData01 │ 5.5T │ 4.1T │ 1.1T │ 79% │ /bigdata01 │ # => │ 12 │ tmpfs │ 1.0M │ 4.0K │ 1020K │ 1% │ /run/credentials/nix-serve.service │ # => │ 13 │ tmpfs │ 6.3G │ 32M │ 6.3G │ 1% │ /run/user/1000 │ # => ╰────┴───────────────────────────────────┴──────┴──────┴───────┴──────┴────────────────────────────────────╯ ``` For an output like from `df` this is probably the most compact way to achieve a nice tabular output. The `str replace` is needed here because one of the column headers has a space in it. ### [Using `from ssv`](https://www.nushell.sh/cookbook/parsing.html\#using-from-ssv) Also the builtin `from` data parser for `ssv` ( _s_ pace _s_ eparated _v_ alues) can be used: ```nu df -h | str replace "Mounted on" Mounted_On | from ssv --aligned-columns --minimum-spaces 1 ``` The output is identical to the previous example. `from ssv` supports several modifying flags to tweak its behaviour. Note we still need to fix the column headers if they contain unexpected spaces. ### [Using `parse`](https://www.nushell.sh/cookbook/parsing.html\#using-parse) How to parse an arbitrary pattern from a string of text into a multi-column table. ```nu cargo search shells --limit 10 | lines | parse "{crate_name} = {version} #{description}" | str trim # => ───┬──────────────┬─────────────────┬──────────────────────────────────────────────────────────────────────────────── # => # │ crate_name │ version │ description # => ───┼──────────────┼─────────────────┼──────────────────────────────────────────────────────────────────────────────── # => 0 │ shells │ "0.2.0" │ Sugar-coating for invoking shell commands directly from Rust. # => 1 │ pyc-shell │ "0.3.0" │ Pyc is a simple CLI application, which allows you to perform shell commands in # => │ │ │ cyrillic and other a… # => 2 │ ion-shell │ "0.0.0" │ The Ion Shell # => 3 │ sheldon │ "0.6.6" │ Fast, configurable, shell plugin manager. # => 4 │ nu │ "0.44.0" │ A new type of shell # => 5 │ git-gamble │ "2.3.0" │ blend TCR + TDD to make sure to develop the right thing, babystep by babystep # => 6 │ martin │ "1.0.0-alpha.0" │ Blazing fast and lightweight PostGIS vector tiles server # => 7 │ fnm │ "1.29.2" │ Fast and simple Node.js version manager # => 8 │ remote_shell │ "2.0.0" │ remote shell written by rust. # => 9 │ sauce │ "0.6.6" │ A tool for managing directory-specific state. # => ───┴──────────────┴─────────────────┴──────────────────────────────────────────────────────────────────────────────── ``` <|firecrawl-page-11-lllmstxt|> ## Nushell vs jq Guide # [jq vs Nushell](https://www.nushell.sh/cookbook/jq_v_nushell.html\#jq-vs-nushell) Both [`jq`](https://jqlang.github.io/jq/) and `nu` have the ability to transform data in a composable way. This cookbook will walk you through common data manipulation tasks with the aim of building a solid mental model for using Nushell effectively. All examples will stick to JSON to keep parity between examples. ## [Consuming JSON](https://www.nushell.sh/cookbook/jq_v_nushell.html\#consuming-json) Let's start with the basics: consuming a JSON string. In `jq`, inputs are always expected to be JSON so we simply do: ```sh echo '{"title": "jq vs Nushell", "publication_date": "2023-11-20"}' | jq -r '.' ``` In `nu`, we need to be explicit because Nushell has a wider range of input choices: ```nu '{"title": "jq vs Nushell", "publication_date": "2023-11-20"}' | from json # => ╭──────────────────┬───────────────╮ # => │ title │ jq vs Nushell │ # => │ publication_date │ 2023-11-20 │ # => ╰──────────────────┴───────────────╯ ``` The output for `jq` is a JSON string whereas in `nu` it's a Nushell value. To get the output of any pipeline as JSON, simply apply a [`to json`](https://www.nushell.sh/commands/docs/to_json.html) at the end: ```nu '[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]' | from json | to json ``` Output: ```json { "title": "jq vs Nushell", "publication_date": "2023-11-20" } ``` When your JSON data is stored in a file, you can use [open](https://www.nushell.sh/commands/docs/open.html) instead of [from json](https://www.nushell.sh/commands/docs/from_json.html). Before we get into the examples, the following glossary can help familiarise yourself with how Nushell data types map to jq data types. | Nushell | jq | | --- | --- | | integer | number | | decimal | number | | string | string | | boolean | boolean | | null | null | | list | array | | record | object | | table | not applicable | | command | filter | ## [Basic operations](https://www.nushell.sh/cookbook/jq_v_nushell.html\#basic-operations) ### [Selecting values](https://www.nushell.sh/cookbook/jq_v_nushell.html\#selecting-values) In `jq`, to get the value from an object we do: ```sh echo '{"name": "Alice", "age": 30}' | jq -r '.name' ``` In `nu` we do: ```nu '{"name": "Alice", "age": 30}' | from json | get name # => Alice ``` ### [Filtering lists](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-lists) In `jq`, to filter an array we do: ```sh echo '[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]' | jq -r '.[] | select(.age > 28)' ``` In `nu` we do: ```nu '[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]' | from json | where age > 28 # => ╭───┬───────┬─────╮ # => │ # │ name │ age │ # => ├───┼───────┼─────┤ # => │ 0 │ Alice │ 30 │ # => ╰───┴───────┴─────╯ ``` ### [Mapping over lists](https://www.nushell.sh/cookbook/jq_v_nushell.html\#mapping-over-lists) In `jq`, to map over a list we do: ```sh echo '[1, 2, 3, 4, 5]' | jq -r 'map(. * 2)' ``` In `nu` we do: ```nu '[1, 2, 3, 4, 5]' | from json | each { |x| $x * 2 } # => ╭───┬────╮ # => │ 0 │ 2 │ # => │ 1 │ 4 │ # => │ 2 │ 6 │ # => │ 3 │ 8 │ # => │ 4 │ 10 │ # => ╰───┴────╯ ``` Note that you can rely on the `$in` auto-binding for a slightly more compact block: ```nu '[1, 2, 3, 4, 5]' | from json | each { $in * 2 } ``` ### [Mapping over records](https://www.nushell.sh/cookbook/jq_v_nushell.html\#mapping-over-records) In `jq`, to map over a record we do: ```sh echo '{"items": [{"name": "Apple", "price": 1}, {"name": "Banana", "price": 0.5}]}' | jq -r '.items | map({(.name): (.price * 2)}) | add' ``` In `nu` we do: ```nu '{"items": [{"name": "Apple", "price": 1}, {"name": "Banana", "price": 0.5}]}' | from json | get items | update price {|row| $row.price * 2} # => ╭───┬────────┬───────╮ # => │ # │ name │ price │ # => ├───┼────────┼───────┤ # => │ 0 │ Apple │ 2 │ # => │ 1 │ Banana │ 1.00 │ # => ╰───┴────────┴───────╯ ``` In this case nu does not require creating new records because we can leverage the fact that a list of records is a table. However, in other situations it might be required as we have seen in [Composing records](https://www.nushell.sh/cookbook/jq_v_nushell.html#composing-records). ### [Sorting lists](https://www.nushell.sh/cookbook/jq_v_nushell.html\#sorting-lists) In `jq`, to sort a list we do: ```sh echo '[3, 1, 4, 2, 5]' | jq -r 'sort' ``` In `nu` we do: ```nu '[3, 1, 4, 2, 5]' | from json | sort # => ╭───┬───╮ # => │ 0 │ 1 │ # => │ 1 │ 2 │ # => │ 2 │ 3 │ # => │ 3 │ 4 │ # => │ 4 │ 5 │ # => ╰───┴───╯ ``` ### [Filtering distinct values in a list](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-distinct-values-in-a-list) In `jq`, to filter a list keeping unique values we do: ```sh echo '[1, 2, 2, 3, 4, 4, 5]' | jq -r 'unique' ``` In `nu` we do: ```nu '[1, 2, 2, 3, 4, 4, 5]' | from json | uniq # => ╭───┬───╮ # => │ 0 │ 1 │ # => │ 1 │ 2 │ # => │ 2 │ 3 │ # => │ 3 │ 4 │ # => │ 4 │ 5 │ # => ╰───┴───╯ ``` ### [Combining filters](https://www.nushell.sh/cookbook/jq_v_nushell.html\#combining-filters) In `jq`, to combine filters we do: ```sh echo '[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]' | jq -r '.[] | select(.age > 28) | .name' ``` In `nu` we do: ```nu '[{"name": "Alice", "age": 30}, {"name": "Bob", "age": 25}]' | from json | where age > 28 | get name # => ╭───┬───────╮ # => │ 0 │ Alice │ # => ╰───┴───────╯ ``` ### [Splitting strings](https://www.nushell.sh/cookbook/jq_v_nushell.html\#splitting-strings) In `jq`, to split a string we do: ```sh echo '{"name": "Alice Smith"}' | jq -r '.name | split(" ") | .[0]' ``` In `nu` we do: ```nu '{"name": "Alice Smith"}' | from json | get name | split words | get 0 # => Alice ``` ### [Conditional logic](https://www.nushell.sh/cookbook/jq_v_nushell.html\#conditional-logic) In `jq` to work with `if` expressions we do: ```sh echo '{"name": "Alice", "age": 30}' | jq -r 'if .age > 18 then "Adult" else "Child" end' ``` In `nu` we do: ```nu '{"name": "Alice", "age": 30}' | from json | if $in.age > 18 { "Adult" } else { "Child" } # => Adult ``` ### [Handling `null` values](https://www.nushell.sh/cookbook/jq_v_nushell.html\#handling-null-values) In `jq`, to filter out `null` values we do: ```sh echo '[1, null, 3, null, 5]' | jq -r 'map(select(. != null))' ``` In `nu` we do: ```nu '[1, null, 3, null, 5]' | from json | where { $in != null } # => ╭───┬───╮ # => │ 0 │ 1 │ # => │ 1 │ 3 │ # => │ 2 │ 5 │ # => ╰───┴───╯ ``` Alternatively, you can use [`compact`](https://www.nushell.sh/commands/docs/compact.html): ```nu '[1, null, 3, null, 5]' | from json | compact ``` ### [Formatting output](https://www.nushell.sh/cookbook/jq_v_nushell.html\#formatting-output) In `jq`, to output a formatted string we do: ```sh echo '{"name": "Alice", "age": 30}' | jq -r "Name: \(.name), Age: \(.age)" ``` In `nu` we do: ```nu '{"name": "Alice", "age": 30}' | from json | items { |key, value| ["Name" $value] | str join ": " } | str join ", " # => Name: Alice, Name: 30 ``` This approach is a bit involved but if we [install the full version](https://github.com/nushell/nushell/releases) which includes the _extra commands_ we can benefit from the [`format`](https://www.nushell.sh/commands/docs/format.html): ```nu '{"name": "Alice", "age": 30}' | from json | format "Name: {name}, Age: {age}" ``` ### [Composing records](https://www.nushell.sh/cookbook/jq_v_nushell.html\#composing-records) In `jq`, to compose a new JSON object (akin to a record in Nushell) we do: ```sh echo '{"name": "Alice", "age": 30}' | jq -r '{name: .name, age: (.age + 5)}' ``` In `nu` we do: ```nu '{"name": "Alice", "age": 30}' | from json | {name: $in.name, age: ($in.age + 5)} # => ╭──────┬───────╮ # => │ name │ Alice │ # => │ age │ 35 │ # => ╰──────┴───────╯ ``` ## [Dealing with nested items](https://www.nushell.sh/cookbook/jq_v_nushell.html\#dealing-with-nested-items) ### [Filtering nested items](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-nested-items) In `jq`, to recursively filter a tree structure we do: ```sh echo '{"data": {"value": 42, "nested": {"value": 24}}}' | jq -r '.. | .value?' ``` In `nu`, there is no built-in command to achieve this, however, we can define our own reusable commands. See the [Appendix: Custom commands](https://www.nushell.sh/cookbook/jq_v_nushell.html#appendix-custom-commands) for an implementation of the command `cherry-pick` shown in the example below. ```nu '{"data": {"value": 42, "nested": {"value": 24}}}' | from json | cherry-pick { |x| $x.value? } # => ╭───┬────╮ # => │ 0 │ │ # => │ 1 │ 42 │ # => │ 2 │ 24 │ # => ╰───┴────╯ ``` ### [Filtering nested arrays](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-nested-arrays) In `jq`, to filter nested arrays we do: ```sh echo '{"data": [{"values": [1, 2, 3]}, {"values": [4, 5, 6]}]}' | jq -r '.data[].values[] | select(. > 3)' ``` In `nu` we can take advantage of the fact that [a list of records is in fact a table](https://www.nushell.sh/book/types_of_data.html#tables) and simply do: ```nu '{"data": [{"values": [1, 2, 3]}, {"values": [4, 5, 6]}]}' | from json | get data.values | flatten | where {|x| $x > 3} # => ╭───┬───╮ # => │ 0 │ 4 │ # => │ 1 │ 5 │ # => │ 2 │ 6 │ # => ╰───┴───╯ ``` ### [Flattening nested records](https://www.nushell.sh/cookbook/jq_v_nushell.html\#flattening-nested-records) In `jq`, to flatten all records preserving their path we do: ```sh echo '{"person": {"name": {"first": "Alice", "last": "Smith"}, "age": 30}}' | jq -r 'paths as $p | select(getpath($p) | type != "object") | ($p | join(".")) + " = " + (getpath($p) | tostring)' ``` In `nu`, there is no built-in command to achieve this. See the [Appendix: Custom commands](https://www.nushell.sh/cookbook/jq_v_nushell.html#appendix-custom-commands) for an implementation of the command `flatten record-paths` shown in the example below. ```nu '{"person": {"name": {"first": "Alice", "last": "Smith"}, "age": 30}}' | from json | flatten record-paths # => ╭───┬───────────────────┬───────╮ # => │ # │ path │ value │ # => ├───┼───────────────────┼───────┤ # => │ 0 │ person.name.first │ Alice │ # => │ 1 │ person.name.last │ Smith │ # => │ 2 │ person.age │ 30 │ # => ╰───┴───────────────────┴───────╯ ``` ### [Mapping over nested items](https://www.nushell.sh/cookbook/jq_v_nushell.html\#mapping-over-nested-items) In `jq`, to traverse a tree we can do: ```sh echo '{"data": {"value": 42, "nested": {"value": 24}}}' | jq -r 'recurse | .value? | select(. != null) | { value: (. * 5) } | add' ``` In `nu`, there is no built-in function equivalent to `recurse`. However, we can reuse the solution from [Filtering nested items](https://www.nushell.sh/cookbook/jq_v_nushell.html#filtering-nested-items) to extract the values to manipulate: ```nu '{"data": {"value": 42, "nested": {"value": 24}}}' | from json | cherry-pick { |x| $x.value? } | compact | each { |x| $x * 5 } # => ╭───┬─────╮ # => │ 0 │ 210 │ # => │ 1 │ 120 │ # => ╰───┴─────╯ ``` ### [Filtering and mapping over nested items](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-and-mapping-over-nested-items) In `jq`, to filter and map over a tree we do: ```sh echo '{"data": {"values": [1, 2, 3], "nested": {"values": [4, 5, 6]}}}' | jq -r 'walk(if type == "number" then . * 2 else . end)' ``` In `nu`, there is no built-in function to achieve this. See the [Appendix: Custom commands](https://www.nushell.sh/cookbook/jq_v_nushell.html#appendix-custom-commands) for an implementation of the command `filter-map` shown in the example below. ```nu '{"data": {"values": [1, 2, 3], "nested": {"values": [4, 5, 6]}}}' | from json | filter-map {|value| if ($value | describe) == "int" { $value * 2 } else { $value }} # => ╭──────┬──────────────────────────────────────╮ # => │ │ ╭────────┬─────────────────────────╮ │ # => │ data │ │ │ ╭───┬───╮ │ │ # => │ │ │ values │ │ 0 │ 2 │ │ │ # => │ │ │ │ │ 1 │ 4 │ │ │ # => │ │ │ │ │ 2 │ 6 │ │ │ # => │ │ │ │ ╰───┴───╯ │ │ # => │ │ │ │ ╭────────┬────────────╮ │ │ # => │ │ │ nested │ │ │ ╭───┬────╮ │ │ │ # => │ │ │ │ │ values │ │ 0 │ 8 │ │ │ │ # => │ │ │ │ │ │ │ 1 │ 10 │ │ │ │ # => │ │ │ │ │ │ │ 2 │ 12 │ │ │ │ # => │ │ │ │ │ │ ╰───┴────╯ │ │ │ # => │ │ │ │ ╰────────┴────────────╯ │ │ # => │ │ ╰────────┴─────────────────────────╯ │ # => ╰──────┴──────────────────────────────────────╯ ``` ## [Grouping and aggregating](https://www.nushell.sh/cookbook/jq_v_nushell.html\#grouping-and-aggregating) ### [Grouping records by key](https://www.nushell.sh/cookbook/jq_v_nushell.html\#grouping-records-by-key) In `jq`, to group a list of records by key we do: ```sh echo '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | jq -r 'group_by(.category)' ``` In `nu` we do: ```nu '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | from json | group-by --to-table category # => ╭───┬───────┬──────────────────────────╮ # => │ # │ group │ items │ # => ├───┼───────┼──────────────────────────┤ # => │ 0 │ A │ ╭───┬──────────┬───────╮ │ # => │ │ │ │ # │ category │ value │ │ # => │ │ │ ├───┼──────────┼───────┤ │ # => │ │ │ │ 0 │ A │ 10 │ │ # => │ │ │ │ 1 │ A │ 5 │ │ # => │ │ │ ╰───┴──────────┴───────╯ │ # => │ 1 │ B │ ╭───┬──────────┬───────╮ │ # => │ │ │ │ # │ category │ value │ │ # => │ │ │ ├───┼──────────┼───────┤ │ # => │ │ │ │ 0 │ B │ 20 │ │ # => │ │ │ ╰───┴──────────┴───────╯ │ # => ╰───┴───────┴──────────────────────────╯ ``` Note that `--to-table` was added to Nushell in [version 0.87.0](https://www.nushell.sh/cookbook/blog/2023-11-14-nushell_0_87_0.html). Before that you had to [`transpose`](https://www.nushell.sh/commands/docs/transpose) the record resulting from `group-by` which was substantially slower for large sets. ### [Aggregating grouped values](https://www.nushell.sh/cookbook/jq_v_nushell.html\#aggregating-grouped-values) In `jq`, to aggregate grouped values we do: ```sh echo '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | jq -r 'group_by(.category) | map({category: .[0].category, sum: map(.value) | add})' ``` In `nu` we do: ```nu '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | from json | group-by --to-table category | update items { |row| $row.items.value | math sum } | rename category sum ``` ### [Filtering after aggregating](https://www.nushell.sh/cookbook/jq_v_nushell.html\#filtering-after-aggregating) In `jq`, to filter after aggregating we do: ```sh echo '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | jq -r 'group_by(.category) | map({category: .[0].category, sum: (map(.value) | add)}) | .[] | select(.sum > 17)' ``` In `nu` we do: ```nu '[{"category": "A", "value": 10}, {"category": "B", "value": 20}, {"category": "A", "value": 5}]' | from json | group-by --to-table category | update items { |row| $row.items.value | math sum } | rename category value | where value > 17 # => ╭───┬──────────┬───────╮ # => │ # │ category │ value │ # => ├───┼──────────┼───────┤ # => │ 0 │ B │ 20 │ # => ╰───┴──────────┴───────╯ ``` ### [Custom aggregations](https://www.nushell.sh/cookbook/jq_v_nushell.html\#custom-aggregations) In `jq`, to apply a custom aggregation we do: ```sh echo '[{"value": 10}, {"value": 20}, {"value": 30}]' | jq -r 'reduce .[] as $item (0; . + $item.value)' ``` In `nu` we do: ```nu '[{"value": 10}, {"value": 20}, {"value": 30}]' | from json | reduce -f 0 { |item, acc| $acc + $item.value } # => 60 ``` ## [Other operations](https://www.nushell.sh/cookbook/jq_v_nushell.html\#other-operations) ### [Calculating averages](https://www.nushell.sh/cookbook/jq_v_nushell.html\#calculating-averages) In `jq`, to calculate an average we do: ```sh echo '[{"score": 90}, {"score": 85}, {"score": 95}]' | jq -r 'map(.score) | add / length' ``` In `nu` we do: ```nu '[{"score": 90}, {"score": 85}, {"score": 95}]' | from json | get score | math avg # => 90 ``` ### [Generating histogram bins](https://www.nushell.sh/cookbook/jq_v_nushell.html\#generating-histogram-bins) In `jq`, to calculate bins for a histogram we do: ```sh echo '[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]' | jq -r 'group_by(. / 5 | floor * 5) | map({ bin: .[0], count: length })' ``` In `nu` we do: ```nu '[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]' | from json | group-by --to-table { $in // 5 * 5 } | each { |row| {bin: $row.items.0, count: ($row.items | length)} } # => ╭───┬─────┬───────╮ # => │ # │ bin │ count │ # => ├───┼─────┼───────┤ # => │ 0 │ 1 │ 4 │ # => │ 1 │ 5 │ 5 │ # => │ 2 │ 10 │ 5 │ # => │ 3 │ 15 │ 1 │ # => ╰───┴─────┴───────╯ ``` Note that if what you are after is computing a histogram, you can benefit from the [`histogram`](https://www.nushell.sh/commands/docs/histogram) command. ## [Appendix: Custom commands](https://www.nushell.sh/cookbook/jq_v_nushell.html\#appendix-custom-commands) This section provides the implementation of the custom commands used in this cookbook. Note that they are illustrative and in no way optimised for large inputs. If you are interested in that, [plugins](https://www.nushell.sh/book/plugins.html) will likely be the answer as they can be written in general purpose languages such as Rust or Python. ```nu use toolbox.nu * help commands | where command_type == "custom" # => ╭──────┬─────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────╮ # => │ # │ name │ usage │ # => ├──────┼─────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────────────────┤ # => │ 0 │ cherry-pick │ A command for cherry-picking values from a record key recursively │ # => │ 1 │ filter-map │ A command for walking through a complex data structure and transforming its values recursively │ # => │ 2 │ flatten record-paths │ A command for flattening trees whilst keeping paths as keys │ # => ╰──────┴─────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` ```nu # toolbox.nu use std/assert # A command for cherry-picking values from a record key recursively export def cherry-pick [\ test # The test function to run over each element\ list: list = [] # The initial list for collecting cherry-picked values\ ] { let input = $in if ($input | describe) =~ "record|table" { $input | values | reduce --fold $list { |value, acc| $acc | append [($value | cherry-pick $test)] } | prepend [(do $test $input)] | flatten } else { $list } } #[test] def test_deep_record_with_key [] { assert equal ({data: {value: 42, nested: {value: 442}}} | cherry-pick {|x| $x.value?}) [null 42 442] assert equal ({value: 42, nested: {value: 442, nested: {other: 4442}}} | cherry-pick {|x| $x.value?}) [42 442 null] assert equal ({ value: 1, nested: {value: 2, nested: {terminal: 3}} terminal: 4, nested2: {value: 5}} | cherry-pick {|x| $x.value?}) [1 2 null 5] } #[test] def test_record_without_key [] { assert equal ({data: 1} | cherry-pick {|x| $x.value?}) [null] } #[test] def test_integer [] { assert equal (1 | cherry-pick {|x| $x.value?}) [] } def test_string [] { assert equal ("foo" | cherry-pick {|x| $x.value?}) [] } #[test] def test_list [] { assert equal (["foo"] | cherry-pick {|x| $x.value?}) [] } #[test] def test_table [] { assert equal ([[a b]; [1.1 1.2] [2.1 2.2]] | cherry-pick {|x| $x.value?}) [null null] assert equal ([[a b]; [1.1 1.2] [2.1 2.2]] | cherry-pick {|x| $x.b?}) [1.2 2.2] } #[test] def test_record_with_key [] { assert equal ({value: 42} | cherry-pick {|x| $x.value?}) [42] assert equal ({value: null} | cherry-pick {|x| $x.value?}) [null] } #[test] def test_deep_record_without_key [] { assert equal ({data: {v: 42}} | cherry-pick {|x| $x.value?}) [null null] } # Like `describe` but dropping item types for collections. export def describe-primitive []: any -> string { $in | describe | str replace --regex '<.*' '' } # A command for cherry-picking values from a record key recursively export def "flatten record-paths" [\ --separator (-s): string = "." # The separator to use when chaining paths\ ] { let input = $in if ($input | describe) !~ "record" { error make {msg: "The record-paths command expects a record"} } $input | flatten-record-paths $separator } def flatten-record-paths [separator: string, ctx?: string] { let input = $in match ($input | describe-primitive) { "record" => { $input | items { |key, value| let path = if $ctx == null { $key } else { [$ctx $key] | str join $separator } {path: $path, value: $value} } | reduce -f [] { |row, acc| $acc | append ($row.value | flatten-record-paths $separator $row.path) | flatten } }, "list" => { $input | enumerate | each { |e| {path: ([$ctx $e.index] | str join $separator), value: $e.item} } }, "table" => { $input | enumerate | each { |r| $r.item | flatten-record-paths $separator ([$ctx $r.index] | str join $separator) } } "block" | "closure" => { error make {msg: "Unexpected type"} }, _ => { {path: $ctx, value: $input} }, } } #[test] def test_record_path [] { assert equal ({a: 1} | flatten record-paths) [{path: "a", value: 1}] assert equal ({a: 1, b: [2 3]} | flatten record-paths) [[path value]; [a 1] ["b.0" 2] ["b.1" 3]] assert equal ({a: 1, b: {c: 2}} | flatten record-paths) [[path value]; [a 1] ["b.c" 2]] assert equal ({a: {b: {c: null}}} | flatten record-paths -s "->") [[path value]; ["a->b->c" null]] } # A command for walking through a complex data structure and transforming its values recursively export def filter-map [mapping_fn: closure] { let input = $in match ($input | describe-primitive) { "record" => { $input | items { |key, value| {key: $key, value: ($value | filter-map $mapping_fn)} } | transpose -rd }, "list" => { $input | each { |value| $value | filter-map $mapping_fn } }, "table" | "block" | "closure" => { error make {msg: "unimplemented"} }, _ => { do $mapping_fn $input }, } } #[test] def test_filtermap [] { assert equal ({a: 42} | filter-map {|x| if ($x | describe) == "int" { $x * 2 } else { $x }}) {a: 84} assert equal ({a: 1, b: 2, c: {d: 3}} | filter-map {|x| if ($x | describe) == "int" { $x * 2 } else { $x }}) {a: 2, b: 4, c: {d: 6}} assert equal ({a: 1, b: "2", c: {d: 3}} | filter-map {|x| if ($x | describe) == "int" { $x * 2 } else { $x }}) {a: 2, b: "2", c: {d: 6}} } ``` ## [Credits](https://www.nushell.sh/cookbook/jq_v_nushell.html\#credits) All jq examples were taken from the [The Ultimate Interactive JQ Guide](https://ishan.page/blog/2023-11-06-jq-by-example/). <|firecrawl-page-12-lllmstxt|> ## File Manipulation Techniques # [Files](https://www.nushell.sh/cookbook/files.html\#files) ### [Editing a file and then saving the changes](https://www.nushell.sh/cookbook/files.html\#editing-a-file-and-then-saving-the-changes) Here we are making edits to `Cargo.toml`. We increase the patch version of the crate using `inc` and then save it back to the file. Use `help inc` to get more information. Read the file's initial contents ```nu open Cargo.toml | get package.version ``` Output `0.59.0` Make the edit to the version number and save it. _Note: running this command should work but it will reorder the toml file alphabetically by section._ ```nu open Cargo.toml | upsert package.version { |p| $p | get package.version | inc --patch } | save -f Cargo.toml ``` Note: `inc` is available through the plugin `nu_plugin_inc`. Output _none_ View the changes we made to the file. ```nu open Cargo.toml | get package.version ``` Output `0.59.1` * * * ### [Parsing a file in a non-standard format](https://www.nushell.sh/cookbook/files.html\#parsing-a-file-in-a-non-standard-format) Suppose you have a file with the following format. ```text band:album:year Fugazi:Steady Diet of Nothing:1991 Fugazi:The Argument:2001 Fugazi:7 Songs:1988 Fugazi:Repeater:1990 Fugazi:In On The Kill Taker:1993 ``` You can parse it into a table. ```nu open bands.txt | lines | split column ":" Band Album Year | skip 1 | sort-by Year # => ━━━┯━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━ # => # │ Band │ Album │ Year # => ───┼────────┼────────────────────────┼────── # => 0 │ Fugazi │ 7 Songs │ 1988 # => 1 │ Fugazi │ Repeater │ 1990 # => 2 │ Fugazi │ Steady Diet of Nothing │ 1991 # => 3 │ Fugazi │ In On The Kill Taker │ 1993 # => 4 │ Fugazi │ The Argument │ 2001 # => ━━━┷━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━ ``` You can alternatively do this using `parse`. ```nu open bands.txt | lines | parse "{Band}:{Album}:{Year}" | skip 1 | sort-by Year ``` Or, you can utilize the `headers` command to use the first row as a header row. The only difference would be the headers would match the case of the text file. So, in this case, the headers would be lowercase. ```nu open bands.txt | lines | split column ":" | headers | sort-by year ``` * * * ### [Word occurrence count with Ripgrep](https://www.nushell.sh/cookbook/files.html\#word-occurrence-count-with-ripgrep) Suppose you would like to check the number of lines the string "Value" appears per file in the nushell project, then sort those files by largest line count. ```nu rg -c Value | lines | split column ":" file line_count | into int line_count | sort-by line_count | reverse # => ───┬──────────────────────────────────────┬──────────── # => # │ file │ line_count # => ───┼──────────────────────────────────────┼──────────── # => 0 │ crates/nu-source/src/meta.rs │ 27 # => 1 │ crates/nu-protocol/src/value/dict.rs │ 10 # => 2 │ src/commands/config.rs │ 10 # => 3 │ crates/nu_plugin_sys/src/sys.rs │ 10 # => 4 │ src/commands/from_bson.rs │ 9 # => 5 │ src/utils/data_processing.rs │ 9 # => 6 │ src/deserializer.rs │ 8 # => 7 │ src/commands/histogram.rs │ 7 # => 8 │ src/commands/split_column.rs │ 6 # => 9 │ src/data/dict.rs │ 6 # => ───┴──────────────────────────────────────┴──────────── # => ... example output limited due to large output ``` <|firecrawl-page-13-lllmstxt|> ## Direnv Configuration Guide # [Direnv](https://www.nushell.sh/cookbook/direnv.html\#direnv) Many people use [direnv](https://direnv.net/) to load an environment upon entering a directory as well as unloading it when exiting the directory. Configuring direnv to work with nushell requires nushell version 0.66 or later. * * * ### [Configuring direnv](https://www.nushell.sh/cookbook/direnv.html\#configuring-direnv) To make direnv work with nushell the way it does with other shells, we can use the "hooks" functionality: ```nu $env.config = { hooks: { pre_prompt: [{ ||\ if (which direnv | is-empty) {\ return\ }\ \ direnv export json | from json | default {} | load-env\ if 'ENV_CONVERSIONS' in $env and 'PATH' in $env.ENV_CONVERSIONS {\ $env.PATH = do $env.ENV_CONVERSIONS.PATH.from_string $env.PATH\ }\ }] } } ``` Note you can follow the [`nu_scripts` of Nushell](https://github.com/nushell/nu_scripts/blob/main/nu-hooks/nu-hooks/direnv/config.nu) for the always up-to-date version of the hook above With that configuration in place, direnv should now work with nushell. <|firecrawl-page-14-lllmstxt|> ## Working with Foreign Shell Scripts # [Working With Foreign Shell Scripts](https://www.nushell.sh/cookbook/foreign_shell_scripts.html\#working-with-foreign-shell-scripts) A common issue with nu is, that other applications export environment variables or functionality as shell scripts, that are expected to then be evaluated by your shell. But many applications only consider the most commonly used shells like `bash` or `zsh`. Unfortunately, nu has entirely incompatible syntax with these shells, so it cannot run or `source` these scripts directly. Generally nothing stops you from running a `zsh` script by invoking `zsh` itself (given it is installed). But unfortunately this will not allow nu to access exported environment variables: ```nu # This works, using zsh to print "Hello" 'echo Hello' | zsh -c $in # This exits with an error because $env.VAR is not defined 'export VAR="Hello"' | zsh -c $in print $env.VAR ``` This chapter presents two workarounds for getting around this issue, and the involved drawbacks. * * * ## [Parsing a Script as a String](https://www.nushell.sh/cookbook/foreign_shell_scripts.html\#parsing-a-script-as-a-string) A naive workaround to extract environment variable declarations is to read the foreign script as a string and parse anything that looks like a variable declaration, so it can be loaded into nushell's environment. ```nu let bash_greeting = ' export GREETING="Hello"; export FROM="from bash"; ' load-env ( $bash_greeting | str trim | lines | parse 'export {name}="{value}";' | transpose --header-row --as-record ) print $"($env.GREETING) ($env.FROM)" # "Hello from bash" ``` This is perfectly fine for situations where you are sure of the exact format of the script and can predict parsing edge cases. This quickly gets tricky though, for example when the script is declaring a `PATH` variable that references its previous value ( `export PATH="$PATH:/extra/path";`). There are ways to implement some form of expansion too, but at some point it might make more sense to leave the parsing to the shell it was meant for. ## [Bash Env Plugin](https://www.nushell.sh/cookbook/foreign_shell_scripts.html\#bash-env-plugin) There is a third-party Nu plugin [bash-env](https://github.com/tesujimath/nu_plugin_bash_env) for importing environment variables from Bash format files and pipes. This plugin uses Bash itself to parse the environment definitions, and can therefore cope with arbitrarily complex Bash sources. Warning Please note that the `bash-env` plugin is not supported by the core Nushell team. All issues and requests for support should be directed to its own [issue tracker](https://github.com/tesujimath/nu_plugin_bash_env/issues). ## [Capturing the environment from a foreign shell script](https://www.nushell.sh/cookbook/foreign_shell_scripts.html\#capturing-the-environment-from-a-foreign-shell-script) A more complex approach is to run the script in the shell it is written for and then do some hackery to capture the script's variables afterwards. Note: The shown command assumes a Unix-like operating system, it may also be possible to implement one for Windows that could capture variables from a PowerShell script. ```nu # Returns a record of changed env variables after running a non-nushell script's contents (passed via stdin), e.g. a bash script you want to "source" def capture-foreign-env [\ --shell (-s): string = /bin/sh\ # The shell to run the script in\ # (has to support '-c' argument and POSIX 'env', 'echo', 'eval' commands)\ --arguments (-a): list = []\ # Additional command line arguments to pass to the foreign shell\ ] { let script_contents = $in; let env_out = with-env { SCRIPT_TO_SOURCE: $script_contents } { ^$shell ...$arguments -c ` env echo '' eval "$SCRIPT_TO_SOURCE" echo '' env -0 -u _ -u _AST_FEATURES -u SHLVL` # Filter out known changing variables } | split row '' | { before: ($in | first | str trim | lines) after: ($in | last | str trim | split row (char --integer 0)) } # Unfortunate Assumption: # No changed env var contains newlines (not cleanly parseable) $env_out.after | where { |line| $line not-in $env_out.before } # Only get changed lines | parse "{key}={value}" | transpose --header-row --as-record | if $in == [] { {} } else { $in } } ``` Usage, e.g. in `env.nu`: ```nu # Default usage, running the script with `/bin/sh` load-env (open script.sh | capture-foreign-env) # Running a different shell's script # fish might be elsewhere on your system, if it's in the PATH, `fish` is enough load-env (open script.fish | capture-foreign-env --shell /usr/local/bin/fish) ``` The command runs a foreign shell script and captures the changed environment variables after running the script. This is done by parsing output of the `env` command available on unix-like systems. The shell to execute can be specified and configured using the `--shell` and `--arguments` parameters, the command has been tested using sh (-> bash), bash, zsh, fish, ksh, and dash. Warning A caveat for this approach is that it requires all changed environment variables not to include newline characters, as the UNIX `env` output is not cleanly parseable in that case. Also beware that directly passing the output of `capture-foreign-env` to `load-env` can result in changed variables like `PATH` to become strings again, even if they have been converted to a list before. ### [Detailed Explanation of `capture-foreign-env`](https://www.nushell.sh/cookbook/foreign_shell_scripts.html\#detailed-explanation-of-capture-foreign-env) Let's have a look at the command's signature first: ```nu def capture-foreign-env [\ --shell (-s): string = /bin/sh\ # The shell to run the script in\ # (has to support '-c' argument and POSIX 'env', 'echo', 'eval' commands)\ --arguments (-a): list = []\ # Additional command line arguments to pass to the foreign shell\ ] { let script_contents = $in; # ... } ``` We're declaring a custom command that takes two optional flags: - `--shell` to specify a shell to run the script in, (e.g. `bash`) - `--arguments` to parse further command line arguments to that shell. The actual script is not mentioned here, because it is read using the special `$in` variable that represents anything passed to Standard Input ( `stdin`), e.g. via a pipe. The shell is set to `/bin/sh` by default, because this is often considered the "default" POSIX-compatible shell of UNIX-like systems, e.g. macOS or Linux. It is often not running the original Bourne shell ( `sh`), but linking to a different shell, like `bash`, with some compatibility flags turned on. As such, many "generic" shell scripts to source are compatible with the system' s `/bin/sh`. Now, let's have a look at where the shell is actually run: ```nu let env_out = with-env { SCRIPT_TO_SOURCE: $script_contents } { ^$shell ...$arguments -c ` ... ` } ``` Essentially, this calls the specified shell (using `^` to run the value as a command) with any arguments specified. It also passes `-c` with an inlined script for the shell, which is the syntax to immediately execute a passed script and exit in most shells. The `with-env { SCRIPT_TO_SOURCE: $script_contents }` block defines an additional environment variable with the actual script we want to run. This is used to pass the script in an unescaped string form, where the executing shell is entirely responsible for parsing it. The alternatives would have been: - Passing the script via `-c $script`, but then we couldn't (safely) add our own commands to log out the environment variables after the script ran. - Using string interpolation, but then we would be responsible for fully escaping the script, so that the `eval "($script)"` line doesn't break due to quotation marks. With the variable expansion in the foreign shell, that shell does not need the value to be escaped; just as nu is normally able to pass a string with any contents to a command as a single string argument. - Using a (temporary or existing) file containing the script - This would also work, but seems unnecessary and potentially slower. Then the external shell executes the script we passed: ```bash env echo '' eval "$SCRIPT_TO_SOURCE" echo '' env -u _ -u _AST_FEATURES -u SHLVL ``` These POSIX-shell compatible commands, available in UNIX-like OSes, do the following: 1. Log out all environment variables at the start of the script. These may be different than the ones in nushell, because the shell might have defined variables on startup and all passed-in variables have been serialized to strings by nushell. 2. Log `` to stdout, this is so we later know where the first `env` output stopped. The content of this is arbitrary, but it is verbose to reduce the risk of any env var having this string in its contents. 3. Run the actual shell script in the current context, using `eval`. The double quotes around the variable are necessary to get newlines to be interpreted correctly. 4. Log the "fence" again to stdout so we know where the "after" list of variables starts. 5. Log all environment variables after the script run. We are excluding a few variables here that are commonly changed by a few shells that have nothing to do with the particular script that was run. We then take the script output and save all lines from the `env` output before and after running the passed script, using the `` logs. ```nu # | split row '' | { before: ($in | first | str trim | lines) after: ($in | last | str trim | lines) } ``` Finally, all that is left to do is to take all env-output lines from the "after" output that were not there before, and parse them into a record: ```nu $env_out.after | where { |line| $line not-in $env_out.before } # Only get changed lines | parse "{key}={value}" | transpose --header-row --as-record ``` <|firecrawl-page-15-lllmstxt|> ## Listening for Keypresses # [Acting on keypresses using `input listen`](https://www.nushell.sh/cookbook/input_listen_keys.html\#acting-on-keypresses-using-input-listen) A common "key listening" pattern is to: - Listen for a specific key (or one of a set of keys) to be pressed - Take action depending on which key was pressed - Loop if one of the expected keys wasn't pressed There are several patterns that can accomplish this, each with advantages and disadvantages. You can choose from one of the following patterns that best fits your use-case and coding style: 1. A first attempt might be the following simple loop. This will work for some cases, but a `loop` cannot itself return a _value_: ```nu def run_some_code [] { print "I'm running the code, but I can't return a" print "value because I need to `break` out of the loop." 42 break } print '(a) Run some code (x) Exit' loop { let key = (input listen --types [key]) if ($key.code == 'a') and ($key.modifiers == []) { run_some_code } else if ($key.code == 'x') and ($key.modifiers == []) { print 'User exited' break } else if ($key.code == 'c') and ($key.modifiers == ['keymodifiers(control)']) { print 'Terminated with Ctrl-C' break } else { print "That key wasn't recognized." print 'Press (a) to run some code or (x) to Exit' continue } } ``` 2. If you need to return a value, you can use a mutable variable to hold the key result after the input loop has ended, _then_ return a value based on the captured keypress: ```nu def run_some_code [] { print "I'm running the code and returning 42" 42 } mut key_props = [] print '(a) Run some code (x) Exit' loop { let key = (input listen --types [key]) $key_props = [$key.code $key.modifiers] let valid_keys = [\ [ 'a' [] ]\ [ 'x' [] ]\ [ 'c' ['keymodifiers(control)'] ]\ ] if $key_props in $valid_keys { break } else { print "That key wasn't recognized." print 'Press (a) to run some code or (x) to Exit' continue } } # Act on the captured keypress from the mutable variable if $key_props == [ 'a' [] ] { run_some_code } else if $key_props == [ 'x' [] ] { print 'User exited' } else if $key_props == [ 'c' ['keymodifiers(control)'] ] { print 'Terminated with Ctrl-C' } ``` 3. This version uses a custom command that recursively calls itself until one of the desired keys is pressed. However, keep in mind that Nushell limits the number of recursive calls based on the value of `$env.config.recursion_limit` (default 50). Hold down the `y` key (not monitored) to demonstrate an early exit based on recursion limits. Note that `break` statements are not needed in this version. ```nu def run_some_code [] { print "I'm running the code and returning 42" 42 } print '(a) Run some code (x) Exit' def input_loop [] { let key = (input listen --types [key]) if ($key.code == 'a') and ($key.modifiers == []) { run_some_code } else if ($key.code == 'x') and ($key.modifiers == []) { print 'User exited' } else if ($key.code == 'c') and ($key.modifiers == ['keymodifiers(control)']) { print 'Terminated with Ctrl-C' } else { print "That key wasn't recognized." print 'Press (a) to run some code or (x) to Exit' # Recurse input_loop } } # Start the loop try { input_loop } catch {|e| print ($e.debug)} ``` 4. The `generate` command offers a functional loop alternative, without recursion limits or mutable variables. `generate` can also collect multiple results into a list, and the output is streamed. ```nu def run_some_code [] { print "I'm running the code and returning 42" 42 } print '(a) Run some code (x) Exit' let key_generator = {|_| let key = (input listen --types [key]) if ($key.code == 'a') and ($key.modifiers == []) { # Returning an "out" record without a "next" terminates the loop { out: (run_some_code) } } else if ($key.code == 'x') and ($key.modifiers == []) { print 'User exited' { out: null } } else if ($key.code == 'c') and ($key.modifiers == ['keymodifiers(control)']) { print 'Terminated with Ctrl-C' { out: null } } else { print "That key wasn't recognized." print 'Press (a) to run some code or (x) to Exit' # Next key generation { next: null } } } generate null $key_generator | get 0 ``` ## [Using match statements with a list of keycodes](https://www.nushell.sh/cookbook/input_listen_keys.html\#using-match-statements-with-a-list-of-keycodes) The above examples use `if`/ `else` statements with hard-coded key values. You may find it easier to maintain your code using `match` statements with a list of keycodes and modifiers. Using this technique, the second example above might look like: ```nu def run_some_code [] { print "I'm running the code and returning 42" 42 } let keys = { # [ key.code key.modifiers ] a: [ 'a' [] ] x: [ 'x' [] ] ctrl-c: [ 'c' ['keymodifiers(control)'] ] } mut key = {keycode: '', modifiers: ['']} print '(a) Run some code (x) Exit' loop { $key = (input listen --types [key]) match [$key.code $key.modifiers] { $keymatch if $keymatch == $keys.a => {break} $keymatch if $keymatch == $keys.x => {print 'User exited'; break} $keymatch if $keymatch == $keys.ctrl-c => {print 'Terminated with Ctrl-C'; break} _ => { print "That key wasn't recognized" print 'Press (a) to run some code or (x) to Exit' continue } } } # Act on the captured keypress from the mutable variable match [$key.code $key.modifiers] { $k if $k == $keys.a => {run_some_code} } ``` <|firecrawl-page-16-lllmstxt|> ## NuShell Help Guide # [Help](https://www.nushell.sh/cookbook/help.html\#help) The `help` command is a good way to become familiar with all that Nu has to offer. ### [How to see all supported commands:](https://www.nushell.sh/cookbook/help.html\#how-to-see-all-supported-commands) ```nu help commands ``` * * * ### [Specific information on a command](https://www.nushell.sh/cookbook/help.html\#specific-information-on-a-command) To find more specific information on a command, use `help `. This works for regular commands (i.e. `http`) and subcommands (i.e. `http get`): ```nu help http get # => Fetch the contents from a URL. # => # => Performs HTTP GET operation. # => # => Search terms: network, fetch, pull, request, download, curl, wget # => # => Usage: # => > http get {flags} # => # => Flags: # => -h, --help - Display the help message for this command # => -u, --user - the username when authenticating # => -p, --password - the password when authenticating # => -t, --timeout - timeout period in seconds # => -H, --headers - custom headers you want to add # => -r, --raw - fetch contents as text rather than a table # => # => Signatures: # => | http get -> # => # => Parameters: # => URL : the URL to fetch the contents from # => # => Examples: # => http get content from example.com # => > http get https://www.example.com # => # => http get content from example.com, with username and password # => > http get -u myuser -p mypass https://www.example.com # => # => http get content from example.com, with custom header # => > http get -H [my-header-key my-header-value] https://www.example.com ``` <|firecrawl-page-17-lllmstxt|> ## Advanced Table Workflows # [Advanced table workflows](https://www.nushell.sh/cookbook/tables.html\#advanced-table-workflows) ### [Merging tables of different size](https://www.nushell.sh/cookbook/tables.html\#merging-tables-of-different-size) Examples shown in [`Working with tables`](https://www.nushell.sh/book/working_with_tables.html) work fine when our tables have equal amount of rows but what if we want to merge tables of different sizes? ```nu let first_table = [[a b]; [1 2] [3 4]] let second_table = [[c d]; [5 6]] $first_table | merge $second_table # => ───┬───┬───┬───┬─── # => # │ a │ b │ c │ d # => ───┼───┼───┼───┼─── # => 0 │ 1 │ 2 │ 5 │ 6 # => ───┼───┼───┼───┼─── # => 1 │ 3 │ 4 │ ❎│ ❎ # => ───┴───┴───┴───┴─── ``` Columns `c` and `d` in the second row are empty because our `second` table only contained a single row; Nushell has nothing to fill the remaining rows with. But what if we wanted the smaller table to 'wrap around' and keep filling the rows? For that we can use the [`chunks`](https://www.nushell.sh/commands/docs/chunks.html) command to split the larger table into subtables, merge each of them with the smaller table and then combine the merged tables together using [`flatten`](https://www.nushell.sh/commands/docs/flatten.html) command For example: ```nu let first_table = [[a b]; [1 2] [3 4]] let second_table = [[c d]; [5 6]] $first_table | chunks ($second_table | length) | each { merge $second_table } | flatten # => ───┬───┬───┬───┬─── # => # │ a │ b │ c │ d # => ───┼───┼───┼───┼─── # => 0 │ 1 │ 2 │ 5 │ 6 # => ───┼───┼───┼───┼─── # => 1 │ 3 │ 4 │ 5 │ 6 # => ───┴───┴───┴───┴─── ``` Can we do that with more than two tables? Sure we can! Let's add a third table: ```nu let third_table = [[e f]; [7 8]] ``` We can merge all three tables like this: ```nu $first_table | chunks ($second_table | length) | each { merge $second_table } | flatten | chunks ($third_table | length) | each { merge $third_table } | flatten # => ───┬───┬───┬───┬───┬───┬─── # => # │ a │ b │ c │ d │ e │ f # => ───┼───┼───┼───┼───┼───┼─── # => 0 │ 1 │ 2 │ 5 │ 6 │ 7 │ 8 # => ───┼───┼───┼───┼───┼───┼─── # => 1 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 # => ───┴───┴───┴───┴───┴───┴─── ``` Or as mentioned in the [Cookbook](https://www.nushell.sh/book/working_with_tables.html#merging-tables) we can use the [`reduce`](https://www.nushell.sh/commands/docs/reduce.html) command to merge tables together recursively: ```nu [$first_table $second_table $third_table] | reduce { |elt, acc| $acc | chunks ($elt | length) | each { merge $elt } | flatten } # => ───┬───┬───┬───┬───┬───┬─── # => # │ a │ b │ c │ d │ e │ f # => ───┼───┼───┼───┼───┼───┼─── # => 0 │ 1 │ 2 │ 5 │ 6 │ 7 │ 8 # => ───┼───┼───┼───┼───┼───┼─── # => 1 │ 3 │ 4 │ 5 │ 6 │ 7 │ 8 # => ───┴───┴───┴───┴───┴───┴─── ``` <|firecrawl-page-18-lllmstxt|> ## Custom Completers Guide # [Custom Completers](https://www.nushell.sh/cookbook/custom_completers.html\#custom-completers) ## [Zoxide Path Completions](https://www.nushell.sh/cookbook/custom_completers.html\#zoxide-path-completions) [Zoxide](https://github.com/ajeetdsouza/zoxide) allows easily jumping between visited folders in the system. It's possible to autocomplete matching folders with this completer: ```nu def "nu-complete zoxide path" [context: string] { let parts = $context | split row " " | skip 1 { options: { sort: false, completion_algorithm: substring, case_sensitive: false, }, completions: (^zoxide query --list --exclude $env.PWD -- ...$parts | lines), } } def --env --wrapped z [...rest: string@"nu-complete zoxide path"] { __zoxide_z ...$rest } ``` Do note that the above completer probably won't work with multiple keywords because each completion suggestion is a full path. Something like `z nu ` might provide `/home/user/nushell` as a suggestion, and if you select this suggestion, your commandline will be replaced with `z nu /home/user/nushell` rather than `z /home/user/nushell`. Running `z nu /home/user/nushell` will now fail. Below is a more convoluted completer that provides odd-looking suggestions but does work with multiple keywords. ```nu def "nu-complete zoxide path" [context: string] { let parts = $context | str trim --left | split row " " | skip 1 | each { str downcase } let completions = ( ^zoxide query --list --exclude $env.PWD -- ...$parts | lines | each { |dir| if ($parts | length) <= 1 { $dir } else { let dir_lower = $dir | str downcase let rem_start = $parts | drop 1 | reduce --fold 0 { |part, rem_start| ($dir_lower | str index-of --range $rem_start.. $part) + ($part | str length) } { value: ($dir | str substring $rem_start..), description: $dir } } }) { options: { sort: false, completion_algorithm: substring, case_sensitive: false, }, completions: $completions, } } ``` <|firecrawl-page-19-lllmstxt|> ## Polars vs Pandas vs Nushell # [Polars vs Pandas vs Nushell](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#polars-vs-pandas-vs-nushell) A dataframe example based on https://studioterabyte.nl/en/blog/polars-vs-pandas ## [1\. Opening the file and show the shape of the DataFrame](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_1-opening-the-file-and-show-the-shape-of-the-dataframe) ```nu let df = polars open NYCTaxi.csv ``` ```nu $df | polars shape # => ╭───┬─────────┬─────────╮ # => │ # │ rows │ columns │ # => ├───┼─────────┼─────────┤ # => │ 0 │ 1458644 │ 11 │ # => ╰───┴─────────┴─────────╯ ``` ## [2\. Opening the file and show the first 5 rows](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_2-opening-the-file-and-show-the-first-5-rows) ```nu $df | polars first 5 | polars collect # => ╭───┬───────────┬───────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬──────────────┬──────────────╮ # => │ # │ id │ vendor_id │ pickup_dateti │ dropoff_datet │ passenger_cou │ pickup_longit │ pickup_latitu │ dropoff_longi │ dropoff_latit │ store_and_fw │ trip_duratio │ # => │ │ │ │ me │ ime │ nt │ ude │ de │ tude │ ude │ d_flag │ n │ # => ├───┼───────────┼───────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼──────────────┼──────────────┤ # => │ 0 │ id2875421 │ 2 │ 2016-03-14 │ 2016-03-14 │ 1 │ -73.98 │ 40.77 │ -73.96 │ 40.77 │ N │ 455 │ # => │ │ │ │ 17:24:55 │ 17:32:30 │ │ │ │ │ │ │ │ # => │ 1 │ id2377394 │ 1 │ 2016-06-12 │ 2016-06-12 │ 1 │ -73.98 │ 40.74 │ -74.00 │ 40.73 │ N │ 663 │ # => │ │ │ │ 00:43:35 │ 00:54:38 │ │ │ │ │ │ │ │ # => │ 2 │ id3858529 │ 2 │ 2016-01-19 │ 2016-01-19 │ 1 │ -73.98 │ 40.76 │ -74.01 │ 40.71 │ N │ 2124 │ # => │ │ │ │ 11:35:24 │ 12:10:48 │ │ │ │ │ │ │ │ # => │ 3 │ id3504673 │ 2 │ 2016-04-06 │ 2016-04-06 │ 1 │ -74.01 │ 40.72 │ -74.01 │ 40.71 │ N │ 429 │ # => │ │ │ │ 19:32:31 │ 19:39:40 │ │ │ │ │ │ │ │ # => │ 4 │ id2181028 │ 2 │ 2016-03-26 │ 2016-03-26 │ 1 │ -73.97 │ 40.79 │ -73.97 │ 40.78 │ N │ 435 │ # => │ │ │ │ 13:30:55 │ 13:38:10 │ │ │ │ │ │ │ │ # => ╰───┴───────────┴───────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴──────────────┴──────────────╯ ``` ## [3\. Opening the file and get the length of all strings in the "id" column](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_3-opening-the-file-and-get-the-length-of-all-strings-in-the-id-column) ```nu let ids = $df | polars first 5 | polars get id | polars str-lengths $df | polars first 5 | polars append $ids | polars rename id_x vendor_id_length # => ╭───┬───────────┬───────────┬──────────────┬──────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────╮ # => │ # │ id │ vendor_id │ pickup_datet │ dropoff_date │ passenger_c │ pickup_long │ pickup_lati │ dropoff_lon │ dropoff_lat │ store_and_f │ trip_durati │ vendor_id_l │ # => │ │ │ │ ime │ time │ ount │ itude │ tude │ gitude │ itude │ wd_flag │ on │ ength │ # => ├───┼───────────┼───────────┼──────────────┼──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ # => │ 0 │ id2875421 │ 2 │ 2016-03-14 │ 2016-03-14 │ 1 │ -73.98 │ 40.77 │ -73.96 │ 40.77 │ N │ 455 │ 9 │ # => │ │ │ │ 17:24:55 │ 17:32:30 │ │ │ │ │ │ │ │ │ # => │ 1 │ id2377394 │ 1 │ 2016-06-12 │ 2016-06-12 │ 1 │ -73.98 │ 40.74 │ -74.00 │ 40.73 │ N │ 663 │ 9 │ # => │ │ │ │ 00:43:35 │ 00:54:38 │ │ │ │ │ │ │ │ │ # => │ 2 │ id3858529 │ 2 │ 2016-01-19 │ 2016-01-19 │ 1 │ -73.98 │ 40.76 │ -74.01 │ 40.71 │ N │ 2124 │ 9 │ # => │ │ │ │ 11:35:24 │ 12:10:48 │ │ │ │ │ │ │ │ │ # => │ 3 │ id3504673 │ 2 │ 2016-04-06 │ 2016-04-06 │ 1 │ -74.01 │ 40.72 │ -74.01 │ 40.71 │ N │ 429 │ 9 │ # => │ │ │ │ 19:32:31 │ 19:39:40 │ │ │ │ │ │ │ │ │ # => │ 4 │ id2181028 │ 2 │ 2016-03-26 │ 2016-03-26 │ 1 │ -73.97 │ 40.79 │ -73.97 │ 40.78 │ N │ 435 │ 9 │ # => │ │ │ │ 13:30:55 │ 13:38:10 │ │ │ │ │ │ │ │ │ # => ╰───┴───────────┴───────────┴──────────────┴──────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────╯ ``` Here's an alternate approach using `with-column` ```nu $df | polars with-column (polars col id | polars str-lengths | polars as vendor_id_lengths) | polars first 5 | polars collect # => ╭───┬───────────┬───────────┬──────────────┬──────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────╮ # => │ # │ id │ vendor_id │ pickup_datet │ dropoff_date │ passenger_c │ pickup_long │ pickup_lati │ dropoff_lon │ dropoff_lat │ store_and_f │ trip_durati │ vendor_id_l │ # => │ │ │ │ ime │ time │ ount │ itude │ tude │ gitude │ itude │ wd_flag │ on │ ength │ # => ├───┼───────────┼───────────┼──────────────┼──────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤ # => │ 0 │ id2875421 │ 2 │ 2016-03-14 │ 2016-03-14 │ 1 │ -73.98 │ 40.77 │ -73.96 │ 40.77 │ N │ 455 │ 9 │ # => │ │ │ │ 17:24:55 │ 17:32:30 │ │ │ │ │ │ │ │ │ # => │ 1 │ id2377394 │ 1 │ 2016-06-12 │ 2016-06-12 │ 1 │ -73.98 │ 40.74 │ -74.00 │ 40.73 │ N │ 663 │ 9 │ # => │ │ │ │ 00:43:35 │ 00:54:38 │ │ │ │ │ │ │ │ │ # => │ 2 │ id3858529 │ 2 │ 2016-01-19 │ 2016-01-19 │ 1 │ -73.98 │ 40.76 │ -74.01 │ 40.71 │ N │ 2124 │ 9 │ # => │ │ │ │ 11:35:24 │ 12:10:48 │ │ │ │ │ │ │ │ │ # => │ 3 │ id3504673 │ 2 │ 2016-04-06 │ 2016-04-06 │ 1 │ -74.01 │ 40.72 │ -74.01 │ 40.71 │ N │ 429 │ 9 │ # => │ │ │ │ 19:32:31 │ 19:39:40 │ │ │ │ │ │ │ │ │ # => │ 4 │ id2181028 │ 2 │ 2016-03-26 │ 2016-03-26 │ 1 │ -73.97 │ 40.79 │ -73.97 │ 40.78 │ N │ 435 │ 9 │ # => │ │ │ │ 13:30:55 │ 13:38:10 │ │ │ │ │ │ │ │ │ # => ╰───┴───────────┴───────────┴──────────────┴──────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────╯ ``` ## [4\. Opening the file and apply a function to the "trip\_duration" to divide the number by 60 to go from the second value to a minute value](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_4-opening-the-file-and-apply-a-function-to-the-trip-duration-to-divide-the-number-by-60-to-go-from-the-second-value-to-a-minute-value) ```nu $df | polars first 5 | polars with-column ((polars col trip_duration) / 60.0) | polars collect # => ╭───┬───────────┬───────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬──────────────┬──────────────╮ # => │ # │ id │ vendor_id │ pickup_dateti │ dropoff_datet │ passenger_cou │ pickup_longit │ pickup_latitu │ dropoff_longi │ dropoff_latit │ store_and_fw │ trip_duratio │ # => │ │ │ │ me │ ime │ nt │ ude │ de │ tude │ ude │ d_flag │ n │ # => ├───┼───────────┼───────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼──────────────┼──────────────┤ # => │ 0 │ id2875421 │ 2 │ 2016-03-14 │ 2016-03-14 │ 1 │ -73.98 │ 40.77 │ -73.96 │ 40.77 │ N │ 7.58 │ # => │ │ │ │ 17:24:55 │ 17:32:30 │ │ │ │ │ │ │ │ # => │ 1 │ id2377394 │ 1 │ 2016-06-12 │ 2016-06-12 │ 1 │ -73.98 │ 40.74 │ -74.00 │ 40.73 │ N │ 11.05 │ # => │ │ │ │ 00:43:35 │ 00:54:38 │ │ │ │ │ │ │ │ # => │ 2 │ id3858529 │ 2 │ 2016-01-19 │ 2016-01-19 │ 1 │ -73.98 │ 40.76 │ -74.01 │ 40.71 │ N │ 35.40 │ # => │ │ │ │ 11:35:24 │ 12:10:48 │ │ │ │ │ │ │ │ # => │ 3 │ id3504673 │ 2 │ 2016-04-06 │ 2016-04-06 │ 1 │ -74.01 │ 40.72 │ -74.01 │ 40.71 │ N │ 7.15 │ # => │ │ │ │ 19:32:31 │ 19:39:40 │ │ │ │ │ │ │ │ # => │ 4 │ id2181028 │ 2 │ 2016-03-26 │ 2016-03-26 │ 1 │ -73.97 │ 40.79 │ -73.97 │ 40.78 │ N │ 7.25 │ # => │ │ │ │ 13:30:55 │ 13:38:10 │ │ │ │ │ │ │ │ # => ╰───┴───────────┴───────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴──────────────┴──────────────╯ ``` ## [5\. Opening the file and filtering out all rows with a trip duration shorther than 500 seconds](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_5-opening-the-file-and-filtering-out-all-rows-with-a-trip-duration-shorther-than-500-seconds) ```nu $df | polars filter-with ((polars col trip_duration) >= 500) | polars first 5 | polars collect # => ╭───┬───────────┬───────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬───────────────┬──────────────┬──────────────╮ # => │ # │ id │ vendor_id │ pickup_dateti │ dropoff_datet │ passenger_cou │ pickup_longit │ pickup_latitu │ dropoff_longi │ dropoff_latit │ store_and_fw │ trip_duratio │ # => │ │ │ │ me │ ime │ nt │ ude │ de │ tude │ ude │ d_flag │ n │ # => ├───┼───────────┼───────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼───────────────┼──────────────┼──────────────┤ # => │ 0 │ id2377394 │ 1 │ 2016-06-12 │ 2016-06-12 │ 1 │ -73.98 │ 40.74 │ -74.00 │ 40.73 │ N │ 663 │ # => │ │ │ │ 00:43:35 │ 00:54:38 │ │ │ │ │ │ │ │ # => │ 1 │ id3858529 │ 2 │ 2016-01-19 │ 2016-01-19 │ 1 │ -73.98 │ 40.76 │ -74.01 │ 40.71 │ N │ 2124 │ # => │ │ │ │ 11:35:24 │ 12:10:48 │ │ │ │ │ │ │ │ # => │ 2 │ id1324603 │ 2 │ 2016-05-21 │ 2016-05-21 │ 1 │ -73.97 │ 40.80 │ -73.92 │ 40.76 │ N │ 1551 │ # => │ │ │ │ 07:54:58 │ 08:20:49 │ │ │ │ │ │ │ │ # => │ 3 │ id0012891 │ 2 │ 2016-03-10 │ 2016-03-10 │ 1 │ -73.98 │ 40.74 │ -73.97 │ 40.79 │ N │ 1225 │ # => │ │ │ │ 21:45:01 │ 22:05:26 │ │ │ │ │ │ │ │ # => │ 4 │ id1436371 │ 2 │ 2016-05-10 │ 2016-05-10 │ 1 │ -73.98 │ 40.76 │ -74.00 │ 40.73 │ N │ 1274 │ # => │ │ │ │ 22:08:41 │ 22:29:55 │ │ │ │ │ │ │ │ # => ╰───┴───────────┴───────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴───────────────┴──────────────┴──────────────╯ ``` ## [6\. Opening the file, filtering out all the rows with a "Y" store\_and\_fwd\_flag value, group by ID and calculate the mean duration time](https://www.nushell.sh/cookbook/polars_v_pandas_v_nushell.html\#_6-opening-the-file-filtering-out-all-the-rows-with-a-y-store-and-fwd-flag-value-group-by-id-and-calculate-the-mean-duration-time) ```nu $df | polars filter-with ((polars col store_and_fwd_flag) == "N") | polars group-by id | polars agg (polars col trip_duration | polars mean) | polars sort-by id | polars first 5 | polars collect # => ╭───┬───────────┬───────────────╮ # => │ # │ id │ trip_duration │ # => ├───┼───────────┼───────────────┤ # => │ 0 │ id0000001 │ 1105.00 │ # => │ 1 │ id0000003 │ 1046.00 │ # => │ 2 │ id0000005 │ 368.00 │ # => │ 3 │ id0000008 │ 303.00 │ # => │ 4 │ id0000009 │ 547.00 │ # => ╰───┴───────────┴───────────────╯ ```