| status | bash | mksh | ksh | toysh | sush | brush | osh | |
| pass | 49 | 45 | 48 | 40 | 44 | 42 | 52 | |
| ok | 2 | 3 | 0 | 0 | 0 | 0 | 0 | |
| N-I | 0 | 1 | 0 | 0 | 0 | 0 | 0 | |
| BUG | 1 | 3 | 0 | 0 | 0 | 0 | 0 | |
| FAIL | 0 | 0 | 4 | 12 | 8 | 10 | 0 | |
| total | 52 | 52 | 52 | 52 | 52 | 52 | 52 | |
| case | bash | mksh | ksh | toysh | sush | brush | osh | description |
| 0 | pass | pass | pass | pass | pass | pass | pass | zero args: [ ] |
| 1 | pass | pass | pass | FAIL | pass | pass | pass | one arg: [ x ] where x is one of '=' '!' '(' ']' |
| details | ||||||||
| 2 | pass | pass | pass | pass | pass | pass | pass | one arg: empty string is false. Equivalent to -n. |
| 3 | pass | pass | pass | pass | FAIL | pass | pass | -a as unary operator (alias of -e) |
| details | ||||||||
| 4 | pass | pass | pass | pass | pass | pass | pass | two args: -z with = ! ( ] |
| 5 | pass | pass | pass | pass | pass | pass | pass | three args |
| 6 | pass | pass | pass | pass | pass | pass | pass | four args |
| 7 | pass | pass | pass | pass | pass | FAIL | pass | test with extra args is syntax error |
| details | ||||||||
| 8 | pass | pass | pass | FAIL | pass | pass | pass | ] syntax errors |
| details | ||||||||
| 9 | pass | pass | pass | pass | pass | pass | pass | -n |
| 10 | pass | pass | pass | pass | pass | pass | pass | ! -a |
| 11 | pass | pass | pass | pass | pass | pass | pass | -o |
| 12 | pass | pass | pass | pass | pass | pass | pass | ( ) |
| 13 | pass | pass | pass | pass | pass | pass | pass | ( ) ! -a -o with system version of [ |
| 14 | pass | pass | pass | pass | pass | pass | pass | == is alias for = |
| 15 | pass | pass | pass | pass | pass | pass | pass | == and = does not do glob |
| 16 | pass | pass | pass | pass | pass | pass | pass | [ with op variable |
| 17 | pass | pass | pass | pass | pass | FAIL | pass | [ with unquoted empty var |
| details | ||||||||
| 18 | pass | pass | pass | pass | pass | pass | pass | [ compare with literal -f |
| 19 | pass | pass | pass | pass | pass | FAIL | pass | [ '(' foo ] is runtime syntax error |
| details | ||||||||
| 20 | pass | pass | pass | pass | FAIL | pass | pass | -z '>' implies two token lookahead |
| details | ||||||||
| 21 | pass | ok | pass | FAIL | FAIL | pass | pass | operator/operand ambiguity with ] |
| details | details | details | ||||||
| 22 | pass | ok | pass | FAIL | FAIL | pass | pass | operator/operand ambiguity with -a |
| details | details | details | ||||||
| 23 | pass | pass | pass | pass | pass | pass | pass | -d |
| 24 | pass | pass | pass | pass | pass | pass | pass | -x |
| 25 | pass | pass | pass | pass | pass | pass | pass | -r |
| 26 | pass | pass | pass | pass | pass | pass | pass | -w |
| 27 | pass | pass | pass | pass | pass | pass | pass | -k for sticky bit |
| 28 | pass | pass | pass | pass | pass | pass | pass | -h and -L test for symlink |
| 29 | pass | pass | pass | pass | pass | pass | pass | -t 1 for stdout |
| 30 | BUG | pass | pass | pass | pass | FAIL | pass | [ -t invalid ] |
| details | details | |||||||
| 31 | pass | pass | pass | pass | pass | pass | pass | -ot and -nt |
| 32 | pass | BUG | FAIL | pass | pass | FAIL | pass | [ a -eq b ] |
| details | details | details | ||||||
| 33 | pass | pass | pass | pass | pass | pass | pass | test -s |
| 34 | pass | pass | pass | pass | pass | pass | pass | test -b -c -S (block, character, socket) |
| 35 | pass | pass | pass | pass | pass | pass | pass | test -p named pipe |
| 36 | pass | pass | pass | FAIL | pass | pass | pass | -G and -O for effective user ID and group ID |
| details | ||||||||
| 37 | pass | pass | pass | pass | pass | pass | pass | -u for setuid, -g too |
| 38 | pass | N-I | pass | FAIL | FAIL | pass | pass | -v to test variable (bash) |
| details | details | details | ||||||
| 39 | pass | pass | pass | FAIL | FAIL | pass | pass | test -o for options |
| details | details | |||||||
| 40 | pass | pass | pass | pass | pass | pass | pass | -nt -ot |
| 41 | pass | pass | pass | pass | pass | pass | pass | -ef |
| 42 | ok | pass | pass | FAIL | FAIL | FAIL | pass | Overflow error |
| details | details | details | details | |||||
| 43 | pass | pass | pass | pass | pass | pass | pass | Bug regression |
| 44 | pass | pass | pass | pass | pass | pass | pass | test -c |
| 45 | pass | pass | pass | pass | pass | pass | pass | test -S |
| 46 | pass | pass | pass | pass | pass | pass | pass | bug from pnut: negative number $((-1)) |
| 47 | pass | pass | pass | FAIL | pass | FAIL | pass | negative octal numbers, etc. |
| details | details | |||||||
| 48 | pass | pass | pass | FAIL | pass | pass | pass | More negative numbers |
| details | ||||||||
| 49 | pass | BUG | FAIL | FAIL | pass | FAIL | pass | No octal, hex, base N conversion - leading 0 is a regular decimal |
| details | details | details | details | |||||
| 50 | ok | ok | FAIL | FAIL | FAIL | FAIL | pass | Looks like octal, but digit is too big |
| details | details | details | details | details | details | |||
| 51 | pass | BUG | FAIL | pass | pass | FAIL | pass | no recursive arith [ 1+2 -eq 3 ] |
| details | details | details |
320 passed, 5 OK, 1 not implemented, 4 BUG, 34 failed, 0 timeouts, 0 cases skipped
| toysh | 1 one arg: [ x ] where x is one of '=' '!' '(' ']' [toysh stdout] Expected 'status=0\nstatus=0\nstatus=0\nstatus=0\n', got 'status=0\nstatus=0\nstatus=2\nstatus=2\n' stdout: status=0 status=0 status=2 status=2stderr: [: need arg @1 [: need arg @1 |
| sush | 3 -a as unary operator (alias of -e) [sush stdout] Expected 'status=0\nstatus=1\n', got 'status=2\nstatus=2\n' stdout: status=2 status=2stderr: [: ‘-a’: unary operator expected [: ‘-a’: unary operator expected |
| brush | 7 test with extra args is syntax error [brush stdout] Expected 'status=2\nstatus=2\n', got 'status=1\nstatus=1\n' stdout: status=1 status=1stderr: |
| toysh | 8 ] syntax errors [toysh stdout] Expected 'status=2\nstatus=1\nstatus=2\nstatus=2\nstatus=2\n' Got '' [toysh status] Expected 0, got -11 stdout: stderr: |
| brush | 17 [ with unquoted empty var [brush status] Expected 2, got 1 stdout: stderr: |
| brush | 19 [ '(' foo ] is runtime syntax error [brush stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| sush | 20 -z '>' implies two token lookahead [sush stdout] Expected 'true\nfalse\ntrue\n', got 'true\nfalse\n' [sush status] Expected 0, got 2 stdout: true falsestderr: [: ‘>’: binary operator expected |
| mksh | 21 operator/operand ambiguity with ] stdout: status=2stderr: mksh: <stdin>[1]: [: ]: unexpected operator/operand |
| toysh | 21 operator/operand ambiguity with ] [toysh stdout] Expected 'status=0\n', got 'status=2\n' stdout: status=2stderr: [: too many arguments |
| sush | 21 operator/operand ambiguity with ] [sush stdout] Expected 'status=0\n', got 'status=2\n' stdout: status=2stderr: [: extra argument ‘]’ |
| mksh | 22 operator/operand ambiguity with -a stdout: status=2stderr: mksh: <stdin>[1]: [: expression expected |
| toysh | 22 operator/operand ambiguity with -a [toysh stdout] Expected 'status=0\n', got 'status=2\n' stdout: status=2stderr: [: need arg @3 |
| sush | 22 operator/operand ambiguity with -a [sush stdout] Expected 'status=0\n', got 'status=2\n' stdout: status=2stderr: [: missing argument after ‘-a’ |
| bash | 30 [ -t invalid ] stdout: status=1stderr: |
| brush | 30 [ -t invalid ] [brush stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| mksh | 32 [ a -eq b ] stdout: status=0stderr: |
| ksh | 32 [ a -eq b ] [ksh stdout] Expected 'status=2\n', got 'status=0\n' stdout: status=0stderr: |
| brush | 32 [ a -eq b ] [brush stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| toysh | 36 -G and -O for effective user ID and group ID [toysh stdout] Expected 'status=0\nstatus=1\nstatus=0\nstatus=1\n', got 'status=2\nstatus=2\nstatus=2\nstatus=2\n' stdout: status=2 status=2 status=2 status=2stderr: test: too many arguments test: too many arguments test: too many arguments test: too many arguments |
| mksh | 38 -v to test variable (bash) stdout: global=2 global=2 dynamic=2 dynamic=2 dynamic=2stderr: mksh: <stdin>[1]: test: nonexistent: unexpected operator/operand mksh: <stdin>[5]: test: g: unexpected operator/operand mksh: <stdin>[21]: test: f_var: unexpected operator/operand mksh: <stdin>[21]: test: g: unexpected operator/operand mksh: <stdin>[21]: test: nonexistent: unexpected operator/operand |
| toysh | 38 -v to test variable (bash) [toysh stdout] Expected 'global=1\nglobal=0\ndynamic=0\ndynamic=0\ndynamic=1\n' Got 'global=2\nglobal=2\ndynamic=2\ndynamic=2\ndynamic=2\n' stdout: global=2 global=2 dynamic=2 dynamic=2 dynamic=2stderr: test: too many arguments test: too many arguments test: too many arguments test: too many arguments test: too many arguments |
| sush | 38 -v to test variable (bash) [sush stdout] Expected 'global=1\nglobal=0\ndynamic=0\ndynamic=0\ndynamic=1\n' Got 'global=2\nglobal=2\ndynamic=2\ndynamic=2\ndynamic=2\n' stdout: global=2 global=2 dynamic=2 dynamic=2 dynamic=2stderr: test: ‘-v’: unary operator expected test: ‘-v’: unary operator expected test: ‘-v’: unary operator expected test: ‘-v’: unary operator expected test: ‘-v’: unary operator expected |
| toysh | 39 test -o for options [toysh stdout] Expected 'status=1\nstatus=0\nstatus=1\n', got 'status=2\nstatus=2\nstatus=2\n' stdout: status=2 status=2 status=2stderr: test: too many arguments set: bad -o nounset test: too many arguments test: too many arguments |
| sush | 39 test -o for options [sush stdout] Expected 'status=1\nstatus=0\nstatus=1\n', got 'status=2\nstatus=2\nstatus=2\n' stdout: status=2 status=2 status=2stderr: test: ‘-o’: unary operator expected /home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: set: nounset: invalid option name test: ‘-o’: unary operator expected test: ‘-o’: unary operator expected |
| bash | 42 Overflow error stdout: status=1stderr: |
| toysh | 42 Overflow error [toysh stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| sush | 42 Overflow error [sush stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| brush | 42 Overflow error [brush stdout] Expected 'status=2\n', got 'status=1\n' stdout: status=1stderr: |
| toysh | 47 negative octal numbers, etc. [toysh stdout] Expected 'zero=0\noctal=1\nhex=2\nbaseN=2\n', got 'zero=0\noctal=0\nhex=0\nbaseN=2\n' stdout: zero=0 octal=0 hex=0 baseN=2stderr: [: not integer: -64#a |
| brush | 47 negative octal numbers, etc. [brush stdout] Expected 'zero=0\noctal=1\nhex=2\nbaseN=2\n', got 'zero=0\noctal=1\nhex=1\nbaseN=1\n' stdout: zero=0 octal=1 hex=1 baseN=1stderr: |
| toysh | 48 More negative numbers [toysh stdout] Expected 'status=0\nstatus=0\n', got '' [toysh status] Expected 0, got -6 stdout: stderr: munmap_chunk(): invalid pointer |
| mksh | 49 No octal, hex, base N conversion - leading 0 is a regular decimal stdout: 73 -73 status=0 status=0 hex=2 baseN=2stderr: mksh: <stdin>[14]: 0xff: bad number '0xff' mksh: <stdin>[16]: 64#a: bad number '64#a' |
| ksh | 49 No octal, hex, base N conversion - leading 0 is a regular decimal [ksh stdout] Expected '59\n-59\n\nstatus=0\nstatus=0\n\nhex=2\nbaseN=2\n' Got '73\n-73\n\nstatus=0\nstatus=0\n\nhex=2\nbaseN=2\n' stdout: 73 -73 status=0 status=0 hex=2 baseN=2stderr: W: ksh: <stdin>[14]: 0xff: bad number: Invalid argument W: ksh: <stdin>[16]: 64#a: bad number: Invalid argument |
| toysh | 49 No octal, hex, base N conversion - leading 0 is a regular decimal [toysh stdout] Expected '59\n-59\n\nstatus=0\nstatus=0\n\nhex=2\nbaseN=2\n' Got '59\n-59\n\nstatus=1\nstatus=1\n\nhex=0\nbaseN=2\n' stdout: 59 -59 status=1 status=1 hex=0 baseN=2stderr: [: not integer: 64#a |
| brush | 49 No octal, hex, base N conversion - leading 0 is a regular decimal [brush stdout] Expected '59\n-59\n\nstatus=0\nstatus=0\n\nhex=2\nbaseN=2\n' Got '59\n-59\n\nstatus=0\nstatus=0\n\nhex=1\nbaseN=1\n' stdout: 59 -59 status=0 status=0 hex=1 baseN=1stderr: |
| bash | 50 Looks like octal, but digit is too big stdout: status=1 status=1 status=0 status=0stderr: bash: line 1: 083: value too great for base (error token is "083") bash: line 4: -083: value too great for base (error token is "083") |
| mksh | 50 Looks like octal, but digit is too big stdout: 83 status=0 -83 status=0 status=0 status=0stderr: |
| ksh | 50 Looks like octal, but digit is too big [ksh stdout] Expected '', got '83\nstatus=0\n-83\nstatus=0\n\nstatus=0\nstatus=0\n' [ksh status] Expected 1, got 0 stdout: 83 status=0 -83 status=0 status=0 status=0stderr: |
| toysh | 50 Looks like octal, but digit is too big [toysh stdout] Expected '', got 'status=1\nstatus=1\n\nstatus=2\nstatus=2\n' [toysh status] Expected 1, got 0 stdout: status=1 status=1 status=2 status=2stderr: main: line 1: sh: bad math: 083 @ 3 main: line 4: sh: bad math: -083 @ 4 [: not integer: 083 [: not integer: -083 |
| sush | 50 Looks like octal, but digit is too big [sush stdout] Expected '', got 'status=1\nstatus=1\n\nstatus=0\nstatus=0\n' [sush status] Expected 1, got 0 stdout: status=1 status=1 status=0 status=0stderr: /home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 1: 083: value too great for base (error token is "083") /home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: 083: value too great for base (error token is "083") |
| brush | 50 Looks like octal, but digit is too big [brush stdout] Expected '', got 'status=1\nstatus=1\n\nstatus=0\nstatus=0\n' [brush status] Expected 1, got 0 stdout: status=1 status=1 status=0 status=0stderr: [31mERROR[0m error: arithmetic evaluation error: failed to parse expression: ' 083 ' [31mERROR[0m error: arithmetic evaluation error: failed to parse expression: ' -083 ' |
| mksh | 51 no recursive arith [ 1+2 -eq 3 ] stdout: status=0 status=0stderr: |
| ksh | 51 no recursive arith [ 1+2 -eq 3 ] [ksh stdout] Expected 'status=2\nstatus=2\n', got 'status=0\nstatus=0\n' stdout: status=0 status=0stderr: |
| brush | 51 no recursive arith [ 1+2 -eq 3 ] [brush stdout] Expected 'status=2\nstatus=2\n', got 'status=1\nstatus=1\n' stdout: status=1 status=1stderr: |