1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
|
# BP - Bruce's PEG Tool
BP is a parsing expression grammar (PEG) tool for the command line.
It's written in pure C with no dependencies.
## Usage
`bp [flags] <pattern> [<input files>...]`
### Flags
* `-h` `--help` print the usage and quit
* `-v` `--verbose` print verbose debugging info
* `-i` `--ignore-case` perform a case-insensitive match
* `-I` `--inplace` perform replacements or filtering in-place on files
* `-C` `--confirm` during replacement, confirm before each replacement
* `-e` `--explain` print an explanation of the matches
* `-j` `--json` print matches as JSON objects
* `-l` `--list-files` print only filenames containing matches
* `-p` `--pattern <pat>` provide a pattern (equivalent to `bp '\(<pat>)'`)
* `-r` `--replace <replacement>` replace the input pattern with the given replacement
* `-c` `--context <N>` change how many lines of context are printed (`0`: no context, `all`: the whole file, `<N>` matching lines and `<N-1>` lines before/after)
* `-g` `--grammar <grammar file>` use the specified file as a grammar
* `-G` `--git` get filenames from git
See `man ./bp.1` for more details.
## BP Patterns
BP patterns are a mixture of Parsing Expression Grammar and Regular
Expression syntax, with a preference for prefix operators instead of
suffix operators.
Pattern | Meaning
-------------------|---------------------
`pat1 pat2` | `pat1` followed by `pat2`
`pat1 / pat2` | `pat1` if it matches, otherwise `pat2`
`..pat` | Any text up to and including `pat` (except newlines)
`..pat1 % pat2` | Any text up to and including `pat1` (except newlines), skipping over instances of `pat2`
`.` | Any single character (except newline)
`^^` | The start of the input
`^` | The start of a line
`$$` | The end of the input
`$` | The end of a line
`__` | Zero or more whitespace characters (including newlines)
`_` | Zero or more whitespace characters (excluding newlines)
`` `c `` | The literal character `c`
`` `a-z `` | The character range `a` through `z`
`` `a,b `` | The character `a` or the character `b`
`\n`, `\033`, `\x0A`, etc. | An escape sequence character
`\x00-xFF` | An escape sequence range (byte `0x00` through `0xFF` here)
`!pat` | `pat` does not match at the current position
`[pat]` or `pat?` | Zero or one occurrences of `pat` (optional pattern)
`5 pat` | Exactly 5 occurrences of `pat`
`2-4 pat` | Between 2 and 4 occurrences of `pat` (inclusive)
`5+ pat` | 5 or more occurrences of `pat`
`5+ pat % sep` | 5 or more occurrences of `pat`, separated by `sep` (e.g. `0+ int % ","` matches `1,2,3`)
`*pat` | 0 or more occurrences of `pat` (shorthand for `0+pat`)
`+pat` | 1 or more occurrences of `pat` (shorthand for `1+pat`)
`<pat` | `pat` matches just before the current position (backref)
`>pat` | `pat` matches just in front of the current position (lookahead)
`@pat` | Capture `pat` (used for text replacement and backreferences)
`@foo=pat` | Let `foo` be the text of `pat` (used for text replacement and backreferences)
`pat => "replacement"` | Match `pat` and replace it with `replacement`
`(pat1 @keep=pat2) => "@keep"` | Match `pat1` followed by `pat2` and replace it with the text of `pat2`
`pat1==pat2` | `pat1`, assuming `pat2` also matches with the same length
`pat1!=pat2` | `pat1`, unless `pat2` also matches with the same length
`name:pat2` | `name` is defined to mean `pat`
`# line comment` | A line comment
See `man ./bp.1` for more details.
## Grammar Files
BP comes packaged with some pattern definitions that can be useful when parsing
code of different languages. Firstly, there are a handful of general-purpose
patterns like:
Name | Meaning
--------------|--------------------
`string` | A string (either single- or double-quoted)
`parens` | A matched pair of parentheses (`()`)
`braces` | A matched pair of curly braces (`{}`)
`brackets` | A matched pair of square brackets (`[]`)
`anglebraces` | A matched pair of angle braces (`<>`)
`_` | Zero or more whitespace characters (excluding newline)
`__` | Zero or more whitespace characters, including newlines and comments
`Abc` | The characters `a-z` and `A-Z`
`Abc123` | The characters `a-z`, `A-Z`, and `0-9`
`int` | 1 or more numeric characters
`number` | An integer or floating point number
`Hex` | A hexadecimal character
`id` | An identifier
`|` | A word boundary
`^` | Start of a line
`^^` | Start of a file
`$` | End of a line
`$$` | End of a file
As well as these common definitions, BP also comes with a set of
language-specific or domain-specific grammars. These are not full language
grammars, but only implementation of some language-specific features, like
identifier rules (`id`), string syntax, and comment syntax (which affects `__`
and other rules). Some of the languages supported are:
- BP
- C++
- C
- Go
- HTML
- Javascript
- Lisp
- Lua
- Python
- Rust
- shell script
These grammar definitions can be found in [grammars](/grammars). To use a
grammar file, use `bp -g <path-to-file>` or `bp --grammar=<path-to-file>`. Once
BP is installed, however, you can use `bp -g <grammar-name>` directly, and BP
will figure out which grammar you mean (e.g. `bp -g lua ...`). BP first
searches `~/.config/bp/` for any grammar files you keep locally, then searches
`/etc/xdg/bp/` for system-wide grammar files.
Testing for these grammar files (other than `builtins`) is iffy at this point,
so use at your own risk! These grammar files are only approximations of syntax.
## Performance
Currently, `bp` is super slow compared to hyper-optimized regex tools like
`grep` and `ripgrep`. `bp` is **not** matching regular expressions, so this is
not strictly a fair comparison. By definition, regular expressions can be
implemented using finite state machines, which are very efficient. Most regex
tools also add the additional restriction that matches must be within a single
line. `bp` on the other hand, uses parsing expression grammars, which can match
arbitrarily complicated or nested structures, requiring a dynamic call stack
and potentially unbounded memory use. This makes `bp` patterns much more
expressive, but harder to optimize. At this point in time, `bp`'s
implementation also uses a fairly naive virtual machine written in C, which is
not very heavily optimized. As a result, `bp` runs quite fast over thousands of
lines of code, reasonably fast over tens of thousands of lines of code, and
pretty slow over millions of lines of code.
## License
BP is provided under the MIT license with the [Commons Clause](https://commonsclause.com/)
(you can't sell this software without the developer's permission, but you're
otherwise free to use, modify, and redistribute it free of charge).
See [LICENSE](LICENSE) for details.
|
