Results for bugs.test.sh

statusbashmkshkshtoyshsushbrushosh
pass 21171712161726
ok 1500001
N-I 0200000
BUG 5300000
FAIL 00101511100
total27272727272727
casebashmkshkshtoyshsushbrushoshdescription
0pass pass pass pass pass pass pass echo keyword
1pass pass pass pass pass pass pass if/else
2pass pass pass FAIL pass pass pass Turn an array into an integer.
details
3pass ok FAIL FAIL pass pass pass assign readonly -- one line
detailsdetailsdetails
4BUG ok FAIL FAIL FAIL FAIL pass assign readonly -- multiple lines
detailsdetailsdetailsdetailsdetailsdetails
5pass ok FAIL FAIL FAIL FAIL pass assign readonly -- multiple lines -- set -o posix
detailsdetailsdetailsdetailsdetails
6pass pass pass pass pass pass pass unset readonly -- one line
7pass pass pass pass pass pass pass unset readonly -- multiple lines
8pass ok FAIL pass FAIL FAIL pass First word like foo$x() and foo$[1+2] (regression)
detailsdetailsdetailsdetails
9pass ok FAIL pass FAIL FAIL pass Function names
detailsdetailsdetailsdetails
10ok pass pass FAIL FAIL FAIL ok file with NUL byte
detailsdetailsdetailsdetailsdetails
11BUG pass FAIL FAIL FAIL pass pass fastlex: PS1 format string that's incomplete / with NUL byte
detailsdetailsdetailsdetails
12pass pass pass pass FAIL pass pass 'echo' and printf fail on writing to full disk
details
13pass BUG pass FAIL FAIL pass pass other builtins fail on writing to full disk
detailsdetailsdetails
14pass pass pass pass pass pass pass subshell while running a script (regression)
15pass N-I FAIL FAIL pass pass pass for loop (issue #1446)
detailsdetailsdetails
16pass N-I FAIL FAIL pass pass pass for loop 2 (issue #1446)
detailsdetailsdetails
17pass pass pass FAIL pass pass pass autoconf word split (#1449)
details
18pass pass pass pass pass pass pass autoconf arithmetic - relaxed eval_unsafe_arith (#1450)
19BUG BUG FAIL FAIL FAIL FAIL pass command execution $(echo 42 | tee PWNED) not allowed
detailsdetailsdetailsdetailsdetailsdetails
20BUG pass pass FAIL FAIL FAIL pass process sub <(echo 42 | tee PWNED) not allowed
detailsdetailsdetailsdetails
21BUG BUG FAIL FAIL FAIL FAIL pass unset doesn't allow command execution
detailsdetailsdetailsdetailsdetailsdetails
22pass pass pass pass pass pass pass printf integer size bug
23pass pass pass FAIL pass FAIL pass (( status bug
detailsdetails
24pass pass pass pass pass pass pass autotools as_fn_arith bug in configure
25pass pass pass FAIL pass pass pass OSH can use ARGV name
details
26pass pass pass pass pass FAIL pass Crash in {1..10} - issue #2296
details
126 passed, 7 OK, 2 not implemented, 8 BUG, 46 failed, 0 timeouts, 0 cases skipped

Details on runs that didn't PASS

toysh2 Turn an array into an integer.

[toysh stdout] Expected '42\n', got ''
[toysh status] Expected 0, got 2

stdout: 
stderr: 


main: line 1: sh: syntax error: 1

mksh3 assign readonly -- one line


stdout: 


stderr: 


mksh: <stdin>[1]: read-only: x

ksh3 assign readonly -- one line


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[1]: read-only: x

toysh3 assign readonly -- one line


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


sh: readonly: No such file or directory

bash4 assign readonly -- multiple lines


stdout: 


hi

stderr: 


bash: line 2: x: readonly variable

mksh4 assign readonly -- multiple lines


stdout: 


stderr: 


mksh: <stdin>[2]: read-only: x

ksh4 assign readonly -- multiple lines


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[2]: read-only: x

toysh4 assign readonly -- multiple lines


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


sh: readonly: No such file or directory

sush4 assign readonly -- multiple lines


[sush stdout] Expected '', got 'hi\n'

[sush status] Expected 1, got 0



stdout: 


hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: x: readonly variable

brush4 assign readonly -- multiple lines


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 1, got 0



stdout: 


hi

stderr: 


ERROR error: cannot mutate readonly variable

mksh5 assign readonly -- multiple lines -- set -o posix


stdout: 


stderr: 


mksh: <stdin>[3]: read-only: x

ksh5 assign readonly -- multiple lines -- set -o posix


[ksh status] Expected 1, got 2



stdout: 


stderr: 


E: ksh: <stdin>[3]: read-only: x

toysh5 assign readonly -- multiple lines -- set -o posix


[toysh stdout] Expected '', got 'hi\n'

[toysh status] Expected 1, got 0



stdout: 


hi

stderr: 


set: bad -o posix
sh: readonly: No such file or directory

sush5 assign readonly -- multiple lines -- set -o posix


[sush stdout] Expected '', got 'hi\n'

[sush status] Expected 1, got 0



stdout: 


hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 3: x: readonly variable

brush5 assign readonly -- multiple lines -- set -o posix


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 1, got 0



stdout: 


hi

stderr: 


ERROR error: cannot mutate readonly variable

mksh8 First word like foo$x() and foo$[1+2] (regression)


stdout: 


stderr: 


mksh: <stdin>[1]: syntax error: '"z"' unexpected

ksh8 First word like foo$x() and foo$[1+2] (regression)


[ksh status] Expected 2, got 1



stdout: 


stderr: 


E: ksh: <stdin>[1]: syntax error: unexpected '"z"'

sush8 First word like foo$x() and foo$[1+2] (regression)


[sush stdout] Expected '', got 'DONE\n'

[sush status] Expected 2, got 0



stdout: 


DONE

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: Unexpected token: ('z')

/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 3: foo$[1+2]: command not found

brush8 First word like foo$x() and foo$[1+2] (regression)


[brush stdout] Expected '', got 'DONE\n'

[brush status] Expected 2, got 0



stdout: 


DONE

stderr: 


ERROR main: syntax error near token `
' (line 1 col 18)
foo$[1+2]: command not found

mksh9 Function names


stdout: 


stderr: 


mksh: <stdin>[1]: foo$x: invalid function name

ksh9 Function names


[ksh status] Expected 2, got 1



stdout: 


stderr: 


E: ksh: <stdin>[1]: foo$x: invalid function name

sush9 Function names


[sush stdout] Expected '', got 'hi\nhi\n'



stdout: 


hi
hi

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 2: Unexpected token: ()
/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: Unexpected token: }

/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 6: Unexpected token: ()
/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 8: Unexpected token: }


brush9 Function names


[brush stdout] Expected '', got 'hi\n'

[brush status] Expected 2, got 0



stdout: 


hi

stderr: 


ERROR main: syntax error near token `{' (line 1 col 10)
ERROR main: syntax error near token `
' (line 1 col 2)

bash10 file with NUL byte


stdout: 


stderr: 


tmp.sh: tmp.sh: cannot execute binary file

toysh10 file with NUL byte


[toysh stdout] Expected 'one echo two\n', got ''

[toysh status] Expected 0, got 1



stdout: 


stderr: 


tmp.sh: line 1: sh: 'tmp.sh' is binary

sush10 file with NUL byte


[sush stdout] Expected 'one echo two\n', got ''

[sush status] Expected 0, got 101



stdout: 


stderr: 


thread 'main' panicked at src/proc_ctrl.rs:181:46:
called `Result::unwrap()` on an `Err` value: NulError(0, [0])
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

brush10 file with NUL byte


[brush stdout] Expected 'one echo two\n', got 'one \x00 echo two\n'



stdout: 


one  echo two

stderr: 


osh10 file with NUL byte


stdout: 


one

stderr: 


bash11 fastlex: PS1 format string that's incomplete / with NUL byte


stdout: 


stderr: 


ksh11 fastlex: PS1 format string that's incomplete / with NUL byte


[ksh stdout] Expected 'x=\\D{%H:%M\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[4]: ${x@P}: bad substitution

toysh11 fastlex: PS1 format string that's incomplete / with NUL byte


[toysh stdout] Expected 'x=\\D{%H:%M\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

sush11 fastlex: PS1 format string that's incomplete / with NUL byte


[sush stdout] Expected 'x=\\D{%H:%M\n', got ''

[sush status] Expected 0, got 1



stdout: 


stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: `${x@P}': bad substitution

sush12 'echo' and printf fail on writing to full disk


[sush stdout] Expected 'status=1\nstatus=1\n', got 'status=1\n'

[sush status] Expected 0, got 101



stdout: 


status=1

stderr: 


echo: write error: No space left on device

thread 'main' panicked at library/std/src/io/stdio.rs:1165:9:
failed printing to stdout: No space left on device (os error 28)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

mksh13 other builtins fail on writing to full disk


stdout: 


status=0
status=0

stderr: 


toysh13 other builtins fail on writing to full disk


[toysh stdout] Expected 'status=1\nstatus=1\n', got 'status=127\nstatus=127\n'



stdout: 


status=127
status=127

stderr: 


sh: type: No such file or directory
sh: ulimit: No such file or directory

sush13 other builtins fail on writing to full disk


[sush stdout] Expected 'status=1\nstatus=1\n', got ''

[sush status] Expected 0, got 101



stdout: 


stderr: 


thread 'main' panicked at library/std/src/io/stdio.rs:1165:9:
failed printing to stdout: No space left on device (os error 28)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

mksh15 for loop (issue #1446)


stdout: 


stderr: 


ksh15 for loop (issue #1446)


[ksh stdout] Expected '0\n1\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[3]: syntax error: unexpected '(('

toysh15 for loop (issue #1446)


[toysh stdout] Expected '0\n1\n', got ''

[toysh status] Expected 0, got 2



stdout: 


stderr: 


main: line 1: sh: syntax error: (

mksh16 for loop 2 (issue #1446)


stdout: 


stderr: 


ksh16 for loop 2 (issue #1446)


[ksh stdout] Expected '0\n1\n', got ''

[ksh status] Expected 0, got 1



stdout: 


stderr: 


E: ksh: <stdin>[4]: syntax error: unexpected '(('

toysh16 for loop 2 (issue #1446)


[toysh stdout] Expected '0\n1\n', got ''

[toysh status] Expected 0, got 2



stdout: 


stderr: 


main: line 1: sh: syntax error: (

toysh17 autoconf word split (#1449)



[toysh stdout]
Expected '--- $()\n[-n]\n[my sed command]\n[f1]\n[f2]\n--- backticks\n[-n]\n[my sed command]\n[f1]\n[f2]\nNOT SPLIT\n'
Got      '--- $()\n[-n]\n[my sed command]\n[f1]\n[f2]\n--- backticks\n[-n]\n[my sed command]\n[f1]\n[f2]\n'


[toysh status] Expected 0, got 2



stdout: 


--- $()
[-n]
[my sed command]
[f1]
[f2]
--- backticks
[-n]
[my sed command]
[f1]
[f2]

stderr: 


main: line 22: sh: syntax error: (

bash19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


1
42

stderr: 


mksh19 command execution $(echo 42 | tee PWNED) not allowed


stdout: 


1
42

stderr: 


ksh19 command execution $(echo 42 | tee PWNED) not allowed


[ksh stdout] Expected u'', got '1\n42\n'

[ksh status] Expected 1, got 0



stdout: 


1
42

stderr: 


toysh19 command execution $(echo 42 | tee PWNED) not allowed


[toysh stdout] Expected u'', got 'NOPE\n'

[toysh status] Expected 1, got 0



stdout: 


NOPE

stderr: 


main: line 4: sh: bad math: x  @ -12462
main: line 4: sh: bad math:  x  @ 2

sush19 command execution $(echo 42 | tee PWNED) not allowed


[sush stdout] Expected u'', got '1\n42\n'

[sush status] Expected 1, got 0



stdout: 


1
42

stderr: 


brush19 command execution $(echo 42 | tee PWNED) not allowed


[brush stdout] Expected u'', got 'NOPE\n'

[brush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


ERROR error: arithmetic evaluation error: failed to parse expression: 'a[$(echo 42 | tee PWNED)]=1'

bash20 process sub <(echo 42 | tee PWNED) not allowed


stdout: 


NOPE

stderr: 


bash: line 4: <(echo 42 | tee PWNED): syntax error: operand expected (error token is "<(echo 42 | tee PWNED)")

toysh20 process sub <(echo 42 | tee PWNED) not allowed


[toysh stdout] Expected u'', got 'NOPE\n'

[toysh status] Expected 1, got 0



stdout: 


NOPE

stderr: 


main: line 4: sh: bad math: x  @ -12462
main: line 4: sh: bad math:  x  @ 2

sush20 process sub <(echo 42 | tee PWNED) not allowed


[sush stdout] Expected u'', got 'NOPE\n'

[sush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


/home/andy/git/oils-for-unix/oils/../../shells/rusty_bash/target/release/sush: line 4: x : syntax error: operand expected (error token is "<")

brush20 process sub <(echo 42 | tee PWNED) not allowed


[brush stdout] Expected u'', got 'NOPE\n'

[brush status] Expected 1, got 0



stdout: 


NOPE

stderr: 


ERROR error: arithmetic evaluation error: failed to parse expression: 'a[<(echo 42 | tee PWNED)]=1'

bash21 unset doesn't allow command execution


stdout: 


len=1
len=0
PWNED
0

stderr: 


mksh21 unset doesn't allow command execution


stdout: 


len=1
len=0
PWNED
0

stderr: 


ksh21 unset doesn't allow command execution


[ksh stdout] Expected 'len=1\n', got 'len=1\nlen=0\nPWNED\n0\n'

[ksh status] Expected 1, got 0



stdout: 


len=1
len=0
PWNED
0

stderr: 


toysh21 unset doesn't allow command execution


[toysh stdout] Expected 'len=1\n', got ''

[toysh status] Expected 1, got 2



stdout: 


stderr: 


sh: typeset: No such file or directory
main: line 2: sh: syntax error: 42

sush21 unset doesn't allow command execution


[sush stdout] Expected 'len=1\n', got 'len=1\nlen=1\nNOPE\n'

[sush status] Expected 1, got 0



stdout: 


len=1
len=1
NOPE

stderr: 


brush21 unset doesn't allow command execution


[brush stdout] Expected 'len=1\n', got 'len=1\nlen=1\nNOPE\n'

[brush status] Expected 1, got 0



stdout: 


len=1
len=1
NOPE

stderr: 


ERROR failed to parse arithmetic expression

toysh23 (( status bug


[toysh stdout] Expected 'status=0\nyes\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

brush23 (( status bug


[brush stdout] Expected 'status=0\nyes\n', got ''

[brush status] Expected 0, got 2



stdout: 


stderr: 


ERROR main: unterminated here document sequence; tag(s) [32] found at: [line 1 col 11] (detected near line 6 column 1)

toysh25 OSH can use ARGV name


[toysh stdout] Expected 'len=2\n', got ''

[toysh status] Expected 0, got -6



stdout: 


stderr: 


munmap_chunk(): invalid pointer

brush26 Crash in {1..10} - issue #2296


[brush status] Expected 127, got 0



stdout: 


stderr: 


1: command not found