1 % BP(1)
   2 % Bruce Hill (*bruce@bruce-hill.com*)
   3 % May 17 2021
   4 
   5 # NAME
   6 
   7 bp - Bruce\'s Parsing Expression Grammar tool
   8 
   9 # SYNOPSIS
  10 
  11 `bp` \[*options...*\] *pattern* \[\[`--`\] *files...*\]
  12 
  13 # DESCRIPTION
  14 
  15 `bp` is a tool that matches parsing expression grammars using a custom
  16 syntax.
  17 
  18 # OPTIONS
  19 
  20 *pattern*
  21 : The text to search for. The main argument for `bp` is a string literals which
  22   may contain BP syntax patterns. See the **STRING PATTERNS** section below.
  23 
  24 `-w`, `--word` *word*
  25 : Surround a string pattern with word boundaries (equivalent to `bp '{|}word{|}'`)
  26 
  27 `-e`, `--explain`
  28 : Print a visual explanation of the matches.
  29 
  30 `-l`, `--list-files`
  31 : Print only the names of files containing matches instead of the matches
  32 themselves.
  33 
  34 `-c`, `--case`
  35 : Perform pattern matching with case-sensitivity (the default is smart casing, i.e. case-insensitive, unless there are any uppercase letters present).
  36 
  37 `-i`, `--ignore-case`
  38 : Perform pattern matching case-insensitively.
  39 
  40 `-I`, `--inplace`
  41 : Perform filtering or replacement in-place (i.e. overwrite files with new
  42 content).
  43 
  44 `-r`, `--replace` *replacement*
  45 : Replace all occurrences of the main pattern with the given string.
  46 
  47 `-s`, `--skip` *pattern*
  48 : While looking for matches, skip over *pattern* occurrences. This can be
  49 useful for behavior like `bp -s string` (avoiding matches inside string
  50 literals).
  51 
  52 `-g`, `--grammar` *grammar-file*
  53 : Load the grammar from the given file. See the `GRAMMAR FILES` section
  54 for more info.
  55 
  56 `-G`, `--git`
  57 : Use `git` to get a list of files. Remaining file arguments (if any) are
  58 passed to `git --ls-files` instead of treated as literal files.
  59 
  60 `-B`, `--context-before` *N*
  61 : The number of lines of context to print before each match (default: 0). See
  62 `--context` below for details on `none` or `all`.
  63 
  64 `-A`, `--context-after` *N*
  65 : The number of lines of context to print after each match (default: 0). See
  66 `--context` below for details on `none` or `all`.
  67 
  68 
  69 `-C`, `--context` *N*
  70 : The number of lines to print before and after each match (default: 0). If *N*
  71 is `none`, print only the exact text of the matches. If *N* is **"all"**, print
  72 all text before and after each match.
  73 
  74 `-f`, `--format` *fancy*\|*plain*\|*bare*\|*file:line*\|*auto*
  75 : Set the output format. *fancy* includes colors and line numbers, *plain*
  76 prints line numbers with no coloring, *bare* prints only the match text,
  77 *file:line* prints the filename and line number for each match (grep-style),
  78 and *auto* (the default) uses *fancy* formatting when the output is a TTY and
  79 *bare* formatting otherwise.
  80 
  81 `-h`, `--help`
  82 : Print the usage and exit.
  83 
  84 *files...*
  85 : The input files to search. If no input files are provided and data was piped
  86 in, that data will be used instead. If neither are provided, `bp` will search
  87 through all files in the current directory and its subdirectories
  88 (recursively).
  89 
  90 
  91 # STRING PATTERNS
  92 
  93 One of the most common use cases for pattern matching tools is matching plain,
  94 literal strings, or strings that are primarily plain strings, with one or two
  95 patterns. `bp` is designed around this fact. The default mode for bp patterns
  96 is "string pattern mode". In string pattern mode, all characters are
  97 interpreted literally except for curly braces `{}`, which mark a region of BP
  98 syntax patterns (see the **PATTERNS** section below). In other words, when
  99 passing a search query to `bp`, you do not need to escape periods, quotation
 100 marks, backslashes, or any other character, as long as it fits inside a shell
 101 string literal. In order to match a literal `{`, you can either search for the
 102 character literal: ``` {`{} ```, the string literal: `{"{"}`, or a pair of 
 103 matching curly braces using the `braces` rule: `{braces}`.
 104 
 105 
 106 # PATTERNS
 107 
 108 `bp` patterns are based off of a combination of Parsing Expression Grammars and
 109 regular expression syntax. The syntax is designed to map closely to verbal
 110 descriptions of the patterns, and prefix operators are preferred over suffix
 111 operators (as is common in regex syntax). Patterns are whitespace-agnostic, so
 112 they work the same regardless of whether whitespace is present or not, except
 113 for string literals (`'...'` and `"..."`), character literals (`` ` ``), and
 114 escape sequences (`\`). Whitespace between patterns or parts of a pattern
 115 should be used for clarity, but it will not affect the meaning of the pattern.
 116 
 117 *pat1 pat2*
 118 : A sequence: *pat1* followed by *pat2*
 119 
 120 *pat1* `/` *pat2*
 121 : A choice: *pat1*, or if it doesn\'t match, then *pat2*
 122 
 123 `.`
 124 : The period pattern matches single character (excluding newline)
 125 
 126 `^`
 127 : Start of a line
 128 
 129 `^^`
 130 : Start of the text
 131 
 132 `$`
 133 : End of a line (does not include newline character)
 134 
 135 `$$`
 136 : End of the text
 137 
 138 `_`
 139 : Zero or more whitespace characters, including spaces and tabs, but not
 140 newlines.
 141 
 142 `__`
 143 : Zero or more whitespace characters, including spaces, tabs, newlines, and
 144 comments. Comments are undefined by default, but may be defined by a separate
 145 grammar file. See the **GRAMMAR FILES** section for more info.
 146 
 147 `"foo"`, `'foo'`
 148 : The literal string **"foo"**. Single and double quotes are treated the same.
 149 Escape sequences are not allowed.
 150 
 151 `` ` ``*c*
 152 : The literal character *c* (e.g. `` `@ `` matches the "@" character)
 153 
 154 `` ` ``*c1*`-`*c2*
 155 : The character range *c1* to *c2* (e.g. `` `a-z ``). Multiple ranges
 156 can be combined with a comma (e.g. `` `a-z,A-Z ``).
 157 
 158 `` ` ``*c1*`,`*c2*
 159 : Any one of the given character or character ranges *c1* or *c2* (e.g. `` `a,e,i,o,u,0-9 ``)
 160 
 161 `\`*esc*
 162 : An escape sequence (e.g. `\n`, `\x1F`, `\033`, etc.)
 163 
 164 `\`*esc1*`-`*esc2*
 165 : An escape sequence range from *esc1* to *esc2* (e.g. `\x00-x1F`)
 166 
 167 `\`*esc1*`,`*esc2*
 168 : Any one of the given escape sequences or ranges *esc1* or *esc2* (e.g. `\r,n,x01-x04`)
 169 
 170 `\N`
 171 : A special escape that matches a "nodent": one or more newlines followed by
 172 the same indentation that occurs on the current line.
 173 
 174 `\C`
 175 : A special escape that always matches the empty string and replaces it with
 176 the indentation of the line on which it matched. For example, this pattern
 177 would match Bash-style heredocs that start with "<<-FOO" and end with a line
 178 containing only the starting indentation and the string "FOO":
 179 `"<<-" @end=(\C id) ..%\n (^end$)`
 180 
 181 `\i`
 182 : An identifier character (e.g. alphanumeric characters or underscores).
 183 
 184 `\I`
 185 : An identifier character, not including numbers (e.g. alphabetic characters or underscores).
 186 
 187 `|`
 188 : A word boundary (i.e. the edge of a word).
 189 
 190 `\b`
 191 : Alias for `|` (word boundary)
 192 
 193 `(` *pat* `)`
 194 : Parentheses can be used to delineate patterns, as in most languages.
 195 
 196 `!` *pat*
 197 : Not *pat* (don't match if *pat* matches here)
 198 
 199 `[` *pat* `]`
 200 : Maybe *pat* (match zero or one occurrences of *pat*)
 201 
 202 *N* *pat*
 203 : Exactly *N* repetitions of *pat* (e.g. `5 "x"` matches **"xxxxx"**)
 204 
 205 *N* `-` *M* *pat*
 206 : Between *N* and *M* repetitions of *pat* (e.g. `2-3 "x"` matches **"xx"** or
 207   **"xxx"**)
 208 
 209 *N*`+` *pat*
 210 : At least *N* or more repetitions of *pat* (e.g. `2+ "x"` matches **"xx"**,
 211   **"xxx"**, **"xxxx"**, etc.)
 212 
 213 `*` *pat*
 214 : Any *pat*s (zero or more, e.g. `* "x"` matches **""**, **"x"**, **"xx"**,
 215   etc.)
 216 
 217 `+` *pat*
 218 : Some *pat*s (one or more, e.g. `+ "x"` matches **"x"**, **"xx"**, **"xxx"**,
 219   etc.)
 220 
 221 *repeating-pat* `%` *sep*
 222 : *repeating-pat* (see the examples above) separated by *sep* (e.g. `*word %
 223   ","` matches zero or more comma-separated words)
 224 
 225 `..` *pat*
 226 : Any text (except newlines) up to and including *pat*. This is a non-greedy
 227   match and does not span newlines.
 228 
 229 `.. %` *skip* *pat*
 230 : Any text (except newlines) up to and including *pat*, skipping over instances
 231   of *skip* (e.g. `'"' ..%('\' .) '"'` opening quote, up to closing quote,
 232   skipping over backslash followed by a single character). A useful application
 233   of the `%` operator is to skip over newlines to perform multi-line matches,
 234   e.g. `pat1 ..%\n pat2`
 235 
 236 `.. =` *only* *pat*
 237 : Any number of repetitions of the pattern *only* up to and including *pat*
 238   (e.g. `"f" ..=abc "k"` matches the letter "f" followed by some alphabetic
 239   characters and then a "k", which would match "fork", but not "free kit") This
 240   is essentially a "non-greedy" version of `*`, and `.. pat` can be thought of
 241   as the special case of `..=. pat`
 242 
 243 `<` *pat*
 244 : Matches at the current position if *pat* matches immediately before the
 245   current position (lookbehind). **Note:** For fixed-length lookbehinds, this
 246   is quite efficient (e.g. `<(100 "x")`), however this can cause performance
 247   problems with variable-length lookbehinds (e.g. `<("x" 0-100"y")`). Also,
 248   patterns like `^`, `^^`, `$`, and `$$` that match against line/file edges
 249   will match against the edge of the lookbehind window, so they should
 250   generally be avoided in lookbehinds.
 251 
 252 `>` *pat*
 253 : Matches *pat*, but does not consume any input (lookahead).
 254 
 255 `@` *pat*
 256 : Capture *pat*. Captured patterns can be used in replacements.
 257 
 258 `foo`
 259 : The named pattern whose name is **"foo"**. Pattern names come from
 260   definitions in grammar files or from named captures. Pattern names may
 261   contain dashes (`-`), but not underscores (`_`), since the underscore is used
 262   to match whitespace. See the **GRAMMAR FILES** section for more info.
 263 
 264 `@` *name* `:` *pat*
 265 : For the rest of the current chain, define *name* to match whatever *pat*
 266   matches, i.e. a backreference. For example, `` @my-word:word `( my-word `) ``
 267   (matches **"asdf(asdf)"** or **"baz(baz)"**, but not **"foo(baz)"**)
 268 
 269 `@` *name* `=` *pat*
 270 : Let *name* equal *pat* (named capture). Named captures can be used in text
 271   replacements.
 272 
 273 *pat* `=>` `"`*replacement*`"`
 274 : Replace *pat* with *replacement*. Note: *replacement* should be a string
 275   (single or double quoted), and it may contain escape sequences (e.g. `\n`) or
 276   references to captured values: `@0` (the whole of *pat*), `@1` (the first
 277   capture in *pat*), `@`*foo* (the capture named *foo* in *pat*), etc. For
 278   example, `@word _ @rest=(*word % _) => "@rest:\n\t@1"` matches a word
 279   followed by whitespace, followed by a series of words and replaces it with
 280   the series of words, a colon, a newline, a tab, and then the first word.
 281 
 282 *pat1* `~` *pat2*
 283 : Matches when *pat1* matches and *pat2* can be found within the text of that
 284   match. (e.g. `comment ~ "TODO"` matches comments that contain **"TODO"**)
 285 
 286 *pat1* `!~` *pat2*
 287 : Matches when *pat1* matches, but *pat2* can not be found within the text of
 288   that match. (e.g. `comment ~ "IGNORE"` matches only comments that do not
 289   contain **"IGNORE"**)
 290 
 291 *name*`:` *pat1*; *pat2*
 292 : Define *name* to mean *pat1* (pattern definition) inside the pattern *pat2*.
 293   For example, a recursive pattern can be defined and used like this:
 294   `paren-comment: "(*" ..%paren-comment "*)"; paren-comment`
 295 
 296 `@:`*name* `=` *pat*
 297 : Match *pat* and tag it with the given name as metadata.
 298 
 299 *name*`::` *pat*
 300 : Syntactic sugar for *name*`:` `@:`*name*`=`*pat* (define a pattern that also
 301   attaches a metadata tag of the same name)
 302 
 303 `#` *comment*
 304 : A line comment, ignored by BP
 305 
 306 
 307 # GRAMMAR FILES
 308 
 309 **bp** allows loading extra grammar files, which define patterns which may be
 310 used for matching. The **builtins** grammar file is loaded by default, and it
 311 defines a few useful general-purpose patterns. For example, it defines the
 312 **parens** rule, which matches pairs of matching parentheses, accounting for
 313 nested inner parentheses:
 314 
 315 ```
 316 bp 'my_func{parens}'
 317 ```
 318 
 319 BP's builtin grammar file defines a few other commonly used patterns such as:
 320 
 321 - `braces` (matching `{}` pairs), `brackets` (matching `[]` pairs),
 322   `anglebraces` (matching `<>` pairs)
 323 - `string`: a single- or double-quote delimited string, including standard
 324   escape sequences
 325 - `id` or `var`: an identifier (full UTF-8 support)
 326 - `word`: similar to `id`/`var`, but can start with a number
 327 - `Hex`, `hex`, `HEX`: a mixed-case, lowercase, or uppercase hex digit
 328 - `digit`: a digit from 0-9
 329 - `int`: one or more digits
 330 - `number`: an int or floating point literal
 331 - `esc`, `tab`, `nl`, `cr`, `crlf`, `lf`: Shorthand for escape sequences
 332 
 333 **bp** also comes with a few grammar files for common programming languages,
 334 which may be loaded on demand. These grammar files are not comprehensive syntax
 335 definitions, but only some common patterns. For example, the c++ grammar file
 336 contains definitions for `//`-style line comments as well as `/*...*/`-style
 337 block comments. Thus, you can find all comments with the word "TODO" with the
 338 following command:
 339 
 340 ```
 341 bp -g c++ '{comment ~ "TODO"}' *.cpp
 342 ```
 343 
 344 
 345 # EXAMPLES
 346 
 347 Find files containing the literal string "foo.baz" (a string pattern):
 348 ```
 349 ls | bp foo.baz
 350 ```
 351 
 352 Find files ending with ".c" and print the name with the ".c" replaced with ".h":
 353 ```
 354 ls | bp '.c{$}' -r '.h'
 355 ```
 356 
 357 Find the word "foobar", followed by a pair of matching parentheses in the file
 358 *my_file.py*:
 359 ```
 360 bp 'foobar{parens}' my_file.py
 361 ```
 362 
 363 Using the *html* grammar, find all *element*s matching the tag *a* in the file
 364 *foo.html*:
 365 ```
 366 bp -g html '{element ~ (^^"<a ")}' foo.html
 367 ```
 368