Results for bugs.test.sh

statusbashdashashzshmkshkshtoyshsushbrushosh
pass 21121318171712161826
ok 1774500001
N-I 0871200000
BUG 5004300000
FAIL 0000010151190
total27272727272727272727
casebashdashashzshmkshkshtoyshsushbrushoshdescription
0pass pass pass pass pass pass pass pass pass pass echo keyword
1pass pass pass pass pass pass pass pass pass pass if/else
2pass N-I N-I pass pass pass FAIL pass pass pass Turn an array into an integer.
detailsdetailsdetails
3pass ok ok pass ok FAIL FAIL pass pass pass assign readonly -- one line
detailsdetailsdetailsdetailsdetails
4BUG ok ok pass ok FAIL FAIL FAIL FAIL pass assign readonly -- multiple lines
detailsdetailsdetailsdetailsdetailsdetailsdetailsdetails
5pass ok ok pass ok FAIL FAIL FAIL FAIL pass assign readonly -- multiple lines -- set -o posix
detailsdetailsdetailsdetailsdetailsdetailsdetails
6pass ok ok ok pass pass pass pass pass pass unset readonly -- one line
detailsdetailsdetails
7pass ok ok ok pass pass pass pass pass pass unset readonly -- multiple lines
detailsdetailsdetails
8pass pass pass ok ok FAIL pass FAIL FAIL pass First word like foo$x() and foo$[1+2] (regression)
detailsdetailsdetailsdetailsdetails
9pass pass pass BUG ok FAIL pass FAIL FAIL pass Function names
detailsdetailsdetailsdetailsdetails
10ok N-I pass ok pass pass FAIL FAIL FAIL ok file with NUL byte
detailsdetailsdetailsdetailsdetailsdetailsdetails
11BUG N-I N-I N-I pass FAIL FAIL FAIL pass pass fastlex: PS1 format string that's incomplete / with NUL byte
detailsdetailsdetailsdetailsdetailsdetailsdetails
12pass pass pass pass pass pass pass FAIL pass pass 'echo' and printf fail on writing to full disk
details
13pass pass pass BUG BUG pass FAIL FAIL pass pass other builtins fail on writing to full disk
detailsdetailsdetailsdetails
14pass pass pass pass pass pass pass pass pass pass subshell while running a script (regression)
15pass N-I N-I pass N-I FAIL FAIL pass pass pass for loop (issue #1446)
detailsdetailsdetailsdetailsdetails
16pass N-I N-I pass N-I FAIL FAIL pass pass pass for loop 2 (issue #1446)
detailsdetailsdetailsdetailsdetails
17pass pass pass pass pass pass FAIL pass pass pass autoconf word split (#1449)
details
18pass pass pass pass pass pass pass pass pass pass autoconf arithmetic - relaxed eval_unsafe_arith (#1450)
19BUG ok ok BUG BUG FAIL FAIL FAIL FAIL pass command execution $(echo 42 | tee PWNED) not allowed
detailsdetailsdetailsdetailsdetailsdetailsdetailsdetailsdetails
20BUG ok ok pass pass pass FAIL FAIL FAIL pass process sub <(echo 42 | tee PWNED) not allowed
detailsdetailsdetailsdetailsdetailsdetails
21BUG N-I N-I BUG BUG FAIL FAIL FAIL FAIL pass unset doesn't allow command execution
detailsdetailsdetailsdetailsdetailsdetailsdetailsdetailsdetails
22pass pass pass pass pass pass pass pass pass pass printf integer size bug
23pass N-I N-I pass pass pass FAIL pass pass pass (( status bug
detailsdetailsdetails
24pass pass pass pass pass pass pass pass pass pass autotools as_fn_arith bug in configure
25pass N-I N-I pass pass pass FAIL pass pass pass OSH can use ARGV name
detailsdetailsdetails
26pass pass pass pass pass pass pass pass FAIL pass Crash in {1..10} - issue #2296
details
170 passed, 25 OK, 18 not implemented, 12 BUG, 45 failed, 0 timeouts, 0 cases skipped

Details on runs that didn't PASS

dash2 Turn an array into an integer.

stdout: 
stderr: 


dash: 1: Syntax error: "(" unexpected

ash2 Turn an array into an integer.


stdout: 


stderr: 


ash: syntax error: unexpected "("

toysh2 Turn an array into an integer.


[toysh stdout] Expected '42\n', got ''

[toysh status] Expected 0, got 2



stdout: 


stderr: 


main: line 1: sh: syntax error: 1

dash3 assign readonly -- one line


stdout: 


stderr: 


dash: 1: x: is read only

ash3 assign readonly -- one line


stdout: 


stderr: 


ash: x: is read only

mksh3 assign readonly -- one line


stdout: 


stderr: 


mksh: <stdin>[1]: read-only: x

ksh3 assign readonly -- one line


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[1]: read-only: x

toysh3 assign readonly -- one line


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


sh: readonly: No such file or directory

bash4 assign readonly -- multiple lines


stdout: 


hi

stderr: 


bash: line 2: x: readonly variable

dash4 assign readonly -- multiple lines


stdout: 


stderr: 


dash: 2: x: is read only

ash4 assign readonly -- multiple lines


stdout: 


stderr: 


ash: x: is read only

mksh4 assign readonly -- multiple lines


stdout: 


stderr: 


mksh: <stdin>[2]: read-only: x

ksh4 assign readonly -- multiple lines


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[2]: read-only: x

toysh4 assign readonly -- multiple lines


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


sh: readonly: No such file or directory

sush4 assign readonly -- multiple lines


[sush stdout] Expected '', got 'hi\n'

[sush status] Expected 1, got 0



stdout: 


hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: x: readonly variable

brush4 assign readonly -- multiple lines


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 1, got 0



stdout: 


hi

stderr: 


ERROR error: cannot mutate readonly variable

dash5 assign readonly -- multiple lines -- set -o posix


stdout: 


stderr: 


dash: 1: set: Illegal option -o posix

ash5 assign readonly -- multiple lines -- set -o posix


stdout: 


stderr: 


ash: set: line 1: illegal option -o posix
ash: x: is read only

mksh5 assign readonly -- multiple lines -- set -o posix


stdout: 


stderr: 


mksh: <stdin>[3]: read-only: x

ksh5 assign readonly -- multiple lines -- set -o posix


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[3]: read-only: x

toysh5 assign readonly -- multiple lines -- set -o posix


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


set: bad -o posix
sh: readonly: No such file or directory

sush5 assign readonly -- multiple lines -- set -o posix


[sush stdout] Expected '', got 'hi\n'

[sush status] Expected 1, got 0



stdout: 


hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 3: x: readonly variable

brush5 assign readonly -- multiple lines -- set -o posix


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 1, got 0



stdout: 


hi

stderr: 


ERROR error: cannot mutate readonly variable

dash6 unset readonly -- one line


stdout: 


stderr: 


dash: 1: unset: x: is read only

ash6 unset readonly -- one line


stdout: 


stderr: 


ash: unset: line 1: x: is read only

zsh6 unset readonly -- one line


stdout: 


stderr: 


zsh: read-only variable: x

dash7 unset readonly -- multiple lines


stdout: 


stderr: 


dash: 2: unset: x: is read only

ash7 unset readonly -- multiple lines


stdout: 


stderr: 


ash: unset: line 2: x: is read only

zsh7 unset readonly -- multiple lines


stdout: 


stderr: 


zsh: read-only variable: x

zsh8 First word like foo$x() and foo$[1+2] (regression)


stdout: 


stderr: 


zsh: unknown file attribute: z

mksh8 First word like foo$x() and foo$[1+2] (regression)


stdout: 


stderr: 


mksh: <stdin>[1]: syntax error: '"z"' unexpected

ksh8 First word like foo$x() and foo$[1+2] (regression)


[ksh status] Expected 2, got 1



stdout: 


stderr: 


E: ksh: <stdin>[1]: syntax error: unexpected '"z"'

sush8 First word like foo$x() and foo$[1+2] (regression)


[sush stdout] Expected '', got 'DONE\n'

[sush status] Expected 2, got 0



stdout: 


DONE

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: Unexpected token: ('z')

/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 3: foo$[1+2]: command not found

brush8 First word like foo$x() and foo$[1+2] (regression)


[brush stdout] Expected '', got 'DONE\n'

[brush status] Expected 2, got 0



stdout: 


DONE

stderr: 


ERROR main: syntax error near token `
' (line 1 col 18)
foo$[1+2]: command not found

zsh9 Function names


stdout: 


stderr: 


mksh9 Function names


stdout: 


stderr: 


mksh: <stdin>[1]: foo$x: invalid function name

ksh9 Function names


[ksh status] Expected 2, got 1



stdout: 


stderr: 


E: ksh: <stdin>[1]: foo$x: invalid function name

sush9 Function names


[sush stdout] Expected '', got 'hi\nhi\n'



stdout: 


hi
hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: Unexpected token: ()
/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: Unexpected token: }

/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 6: Unexpected token: ()
/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 8: Unexpected token: }


brush9 Function names


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 2, got 0



stdout: 


hi

stderr: 


ERROR main: syntax error near token `{' (line 1 col 10)
ERROR main: syntax error near token `
' (line 1 col 2)

bash10 file with NUL byte


stdout: 


stderr: 


tmp.sh: tmp.sh: cannot execute binary file

dash10 file with NUL byte


stdout: 


stderr: 


tmp.sh: 1: tmp.sh: -e: not found

zsh10 file with NUL byte


stdout: 


one echo two

stderr: 


toysh10 file with NUL byte


[toysh stdout] Expected 'one echo two\n', got ''

[toysh status] Expected 0, got 1



stdout: 


stderr: 


tmp.sh: line 1: sh: 'tmp.sh' is binary

sush10 file with NUL byte


[sush stdout] Expected 'one echo two\n', got ''

[sush status] Expected 0, got 101



stdout: 


stderr: 


thread 'main' panicked at src/proc_ctrl.rs:181:46:
called `Result::unwrap()` on an `Err` value: NulError(0, [0])
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

brush10 file with NUL byte


[brush stdout] Expected 'one echo two\n', got 'one \x00 echo two\n'



stdout: 


one  echo two

stderr: 


osh10 file with NUL byte


stdout: 


one

stderr: 


bash11 fastlex: PS1 format string that's incomplete / with NUL byte


stdout: 


stderr: 


dash11 fastlex: PS1 format string that's incomplete / with NUL byte


stdout: 


stderr: 


dash: 4: Bad substitution

ash11 fastlex: PS1 format string that's incomplete / with NUL byte


stdout: 


stderr: 


ash: syntax error: bad substitution

zsh11 fastlex: PS1 format string that's incomplete / with NUL byte


stdout: 


stderr: 


zsh: bad substitution

ksh11 fastlex: PS1 format string that's incomplete / with NUL byte


[ksh stdout] Expected 'x=\\D{%H:%M\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[4]: ${x@P}: bad substitution

toysh11 fastlex: PS1 format string that's incomplete / with NUL byte


[toysh stdout] Expected 'x=\\D{%H:%M\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

sush11 fastlex: PS1 format string that's incomplete / with NUL byte


[sush stdout] Expected 'x=\\D{%H:%M\n', got ''

[sush status] Expected 0, got 1



stdout: 


stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: `${x@P}': bad substitution

sush12 'echo' and printf fail on writing to full disk


[sush stdout] Expected 'status=1\nstatus=1\n', got 'status=1\n'

[sush status] Expected 0, got 101



stdout: 


status=1

stderr: 


echo: write error: No space left on device

thread 'main' panicked at library/std/src/io/stdio.rs:1165:9:
failed printing to stdout: No space left on device (os error 28)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

zsh13 other builtins fail on writing to full disk


stdout: 


status=0
status=0

stderr: 


mksh13 other builtins fail on writing to full disk


stdout: 


status=0
status=0

stderr: 


toysh13 other builtins fail on writing to full disk


[toysh stdout] Expected 'status=1\nstatus=1\n', got 'status=127\nstatus=127\n'



stdout: 


status=127
status=127

stderr: 


sh: type: No such file or directory
sh: ulimit: No such file or directory

sush13 other builtins fail on writing to full disk


[sush stdout] Expected 'status=1\nstatus=1\n', got ''

[sush status] Expected 0, got 101



stdout: 


stderr: 


thread 'main' panicked at library/std/src/io/stdio.rs:1165:9:
failed printing to stdout: No space left on device (os error 28)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

dash15 for loop (issue #1446)


stdout: 


stderr: 


ash15 for loop (issue #1446)


stdout: 


stderr: 


mksh15 for loop (issue #1446)


stdout: 


stderr: 


ksh15 for loop (issue #1446)


[ksh stdout] Expected '0\n1\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[3]: syntax error: unexpected '(('

toysh15 for loop (issue #1446)


[toysh stdout] Expected '0\n1\n', got ''

[toysh status] Expected 0, got 2



stdout: 


stderr: 


main: line 1: sh: syntax error: (

dash16 for loop 2 (issue #1446)


stdout: 


stderr: 


ash16 for loop 2 (issue #1446)


stdout: 


stderr: 


mksh16 for loop 2 (issue #1446)


stdout: 


stderr: 


ksh16 for loop 2 (issue #1446)


[ksh stdout] Expected '0\n1\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[4]: syntax error: unexpected '(('

toysh16 for loop 2 (issue #1446)


[toysh stdout] Expected '0\n1\n', got ''

[toysh status] Expected 0, got 2



stdout: 


stderr: 


main: line 1: sh: syntax error: (

toysh17 autoconf word split (#1449)



[toysh stdout]
Expected '--- $()\n[-n]\n[my sed command]\n[f1]\n[f2]\n--- backticks\n[-n]\n[my sed command]\n[f1]\n[f2]\nNOT SPLIT\n'
Got      '--- $()\n[-n]\n[my sed command]\n[f1]\n[f2]\n--- backticks\n[-n]\n[my sed command]\n[f1]\n[f2]\n'


[toysh status] Expected 0, got 2



stdout: 


--- $()
[-n]
[my sed command]
[f1]
[f2]
--- backticks
[-n]
[my sed command]
[f1]
[f2]

stderr: 


main: line 22: sh: syntax error: (

bash19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


1
42

stderr: 


dash19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


stderr: 


dash: 4: Illegal number: a[$(echo 42 | tee PWNED)]=1

ash19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


stderr: 


ash: arithmetic syntax error

zsh19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


1
42

stderr: 


mksh19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


1
42

stderr: 


ksh19 command execution $(echo 42 | tee PWNED) not allowed


[ksh stdout] Expected u'', got '1\n42\n'

[ksh status] Expected 1, got 0



stdout: 


1
42

stderr: 


toysh19 command execution $(echo 42 | tee PWNED) not allowed


[toysh stdout] Expected u'', got 'NOPE\n'

[toysh status] Expected 1, got 0



stdout: 


NOPE

stderr: 


main: line 4: sh: bad math: x  @ -12462
main: line 4: sh: bad math:  x  @ 2

sush19 command execution $(echo 42 | tee PWNED) not allowed


[sush stdout] Expected u'', got '1\n42\n'

[sush status] Expected 1, got 0



stdout: 


1
42

stderr: 


brush19 command execution $(echo 42 | tee PWNED) not allowed


[brush stdout] Expected u'', got 'NOPE\n'

[brush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


ERROR error: arithmetic evaluation error: failed to parse expression: 'a[$(echo 42 | tee PWNED)]=1'

bash20 process sub <(echo 42 | tee PWNED) not allowed


stdout: 


NOPE

stderr: 


bash: line 4: <(echo 42 | tee PWNED): syntax error: operand expected (error token is "<(echo 42 | tee PWNED)")

dash20 process sub <(echo 42 | tee PWNED) not allowed


stdout: 


stderr: 


dash: 4: Illegal number: a[<(echo 42 | tee PWNED)]=1

ash20 process sub <(echo 42 | tee PWNED) not allowed


stdout: 


stderr: 


ash: arithmetic syntax error

toysh20 process sub <(echo 42 | tee PWNED) not allowed


[toysh stdout] Expected u'', got 'NOPE\n'

[toysh status] Expected 1, got 0



stdout: 


NOPE

stderr: 


main: line 4: sh: bad math: x  @ -12462
main: line 4: sh: bad math:  x  @ 2

sush20 process sub <(echo 42 | tee PWNED) not allowed


[sush stdout] Expected u'', got 'NOPE\n'

[sush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: x : syntax error: operand expected (error token is "<")

brush20 process sub <(echo 42 | tee PWNED) not allowed


[brush stdout] Expected u'', got 'NOPE\n'

[brush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


ERROR error: arithmetic evaluation error: failed to parse expression: 'a[<(echo 42 | tee PWNED)]=1'

bash21 unset doesn't allow command execution


stdout: 


len=1
len=0
PWNED
0

stderr: 


dash21 unset doesn't allow command execution


stdout: 


stderr: 


dash: 1: typeset: not found
dash: 2: Syntax error: "(" unexpected

ash21 unset doesn't allow command execution


stdout: 


stderr: 


ash: typeset: not found
ash: syntax error: unexpected "("

zsh21 unset doesn't allow command execution


stdout: 


len=1
len=1
PWNED
0

stderr: 


zsh: a: assignment to invalid subscript range

mksh21 unset doesn't allow command execution


stdout: 


len=1
len=0
PWNED
0

stderr: 


ksh21 unset doesn't allow command execution


[ksh stdout] Expected 'len=1\n', got 'len=1\nlen=0\nPWNED\n0\n'

[ksh status] Expected 1, got 0



stdout: 


len=1
len=0
PWNED
0

stderr: 


toysh21 unset doesn't allow command execution


[toysh stdout] Expected 'len=1\n', got ''

[toysh status] Expected 1, got 2



stdout: 


stderr: 


sh: typeset: No such file or directory
main: line 2: sh: syntax error: 42

sush21 unset doesn't allow command execution


[sush stdout] Expected 'len=1\n', got 'len=1\nlen=1\nNOPE\n'

[sush status] Expected 1, got 0



stdout: 


len=1
len=1
NOPE

stderr: 


brush21 unset doesn't allow command execution


[brush stdout] Expected 'len=1\n', got 'len=1\nlen=1\nNOPE\n'

[brush status] Expected 1, got 0



stdout: 


len=1
len=1
NOPE

stderr: 


ERROR failed to parse arithmetic expression

dash23 (( status bug


stdout: 


stderr: 


ash23 (( status bug


stdout: 


stderr: 


toysh23 (( status bug


[toysh stdout] Expected 'status=0\nyes\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

dash25 OSH can use ARGV name


stdout: 


stderr: 


ash25 OSH can use ARGV name


stdout: 


stderr: 


toysh25 OSH can use ARGV name


[toysh stdout] Expected 'len=2\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

brush26 Crash in {1..10} - issue #2296


[brush status] Expected 127, got 0



stdout: 


stderr: 


1: command not found