Functional programming in Bash

Introduction to fun.sh library

L2L Warszawa 2017

About me

Sławomir Śledź

Senior Java Developer

ssledz.github.io

The Talk

Bash

Bash is an sh-compatible command language interpreter that executes commands read from the standard input or from a file.

Strong sides

easy to learn
available out of the box on most linux distributions
piping
flexible i/o redirection
painless concurrency
simple type system (everything is a string)
local variables, read only variables (but who is using them ?)

Weak sides

arrays suck
associative arrays (dictionaries) suck
syntax for if, for, while, case,... statements is hard to remember
many traps for beginners
is slow
global variables
debugging is painful
functions can only return numbers (status code)

Common mistakes

What does echo print ?


                cnt=0; cat /proc/meminfo | while read line; do

                    cnt=$((cnt+1))

                done

                echo $cnt

Common mistakes

Now echo prints correctly the number of lines of the meminfo pseudo file.


                cnt=0; while read line; do

                    cnt=$((cnt+1))

                done < /proc/meminfo

                echo $cnt

Common mistakes

Does it print 'xxx' ?


                echo -n 'xxx' | while read line; do

                    echo $line;

                done

Common mistakes

Now it prints 'xxx' !


                 echo -n 'xxx' | while read line || [ -n "$line" ]; do

                    echo $line;

                 done

What can we do to make programming in bash more pleasant ?

What makes haskel programmers so happy ?

What is functional programming ?

Can we adapt some parts of it in bash ?

Functional programming

In computer science, functional programming is a programming paradigm—a style of building the structure and elements of computer programs—that treats computation as the evaluation of mathematical functions and avoids changing-state and mutable data.

Good parts

easier to understand and predict the behavior of a program
plays extremely nice with the rising multi-core technology (concurrency)
many algorithms can be defined in recursive manner
more concise code (comparing to imperative style)
better readable code - encourage decoupling / modular design (comparing to imperative style)

Good parts...

employs some really nice abstractions to solve problems in very generic way (monads, option, lazy, memo, pattern matching, currying)
make you think about problems in more abstract / mathematical way

Bad parts

real-world programs are not stateless
with immutable data structures you have cheap thread safety at the cost of memory consumption
usually programs are shorter and easier to read except the cases where they are longer and harder to read :) (depends on the experience)
productivity goes up in order to hire very expensive programmers who know how to program in a functional style
it takes some time to be efficient in functional programming

fun-bash a quick start

Available on GitHub bash-fun

download fun.sh and


            #!/bin/bash
            . fun.sh

            seq 1 4 | sum


        #!/bin/bash
        . <(curl -Ls goo.gl/8JeMbR)

        seq 1 4 | sum


            #!/bin/bash
            . <(test -e fun.sh || curl -Ls goo.gl/8JeMbR > fun.sh; cat fun.sh)

            seq 1 4 | sum

fun-bash a quick start...

You can use bash-import to load bash-fun


        #!/bin/bash

        BASH_REP=https://raw.githubusercontent.com/ssledz/bash-repo/master

        . <(curl -s $BASH_REP/__import.sh 2>/dev/null)

        import pl.softech/fun/1.0.0 as f

        seq 1 4 | f.sum

Available on GitHub bash-import

Example of usage


            #!/bin/bash
            . fun.sh
            backup_dir=/home/adserver/backup
            max_backups=10 # adserver-active-1080-2017-05-22T134440

            prepare() {
                local a=$1; echo $a
                echo $a | strip adserver-active- | strip 1081- | strip 1080-
            }

            ls -1 $backup_dir | grep adserver-active- \
                | map λ a . 'prepare $a' \
                | buff λ a b . 'date=$(echo $b | cut -dT -f1); tup $a $date' \
                | sort -t ',' -k2 -r | drop $max_backups \
                | map λ a . 'echo -n $backup_dir/; tupl $a' | xargs rm -rf

Example of usage - test data


#!/bin/bash
xargs mkdir << EOF
  adserver-active-1080-2017-05-22T134440
  adserver-active-1080-2017-05-23T143705
  adserver-active-1080-2017-05-29T110500
  adserver-active-1080-2017-05-30T110812
  adserver-active-1080-2017-06-20T130354
EOF

List of available functions

v1.1.1

add	append	buff	curry	div	drop
factorial	filter	foldl	foldr	head	join
lambda	last	list	map	mod	mul
prepend	product	ret	revers_str	revers	scanl
splitc	strip	sub	sum	tail	take
try	tupl	tupr	tup	tupx	unlist
zip	λ

list/unlist


            $ list 1 2 3 4 5
            1
            2
            3
            4
            5
            $ list 1 2 3 4 5 | unlist
            1 2 3 4 5

take/drop/tail/head/last


            $ list 1 2 3 4 | drop 2
            3
            4
            $ list 1 2 3 4 5 | head
            1
            $ list 1 2 3 4 | tail
            2
            3
            4
            $ list 1 2 3 4 5 | last
            5
            $ list 1 2 3 4 5 | take 2
            1
            2

join


            $ list 1 2 3 4 5 | join ,

            1,2,3,4,5

            $ list 1 2 3 4 5 | join , [ ]

            [1,2,3,4,5]

map


        $ seq 1 5 | map lambda a . 'echo $((a + 5))'
        6
        7
        8
        9
        10
        $ list a b s d e | map lambda a . 'echo $a$(echo $a | tr a-z A-Z)'
        aA
        bB
        sS
        dD
        eE

flat map


        $ seq 2 3 | map lambda a . 'seq 1 $a' | join , [ ]

        [1,2,1,2,3]

        $ list a b c | map lambda a . \
            'echo $a; echo $a | tr a-z A-z' | join , [ ]

        [a,A,b,B,c,C]

filter


            $ seq 1 10 | filter lambda a .\
                '[[ $(mod $a 2) -eq 0 ]] && ret true || ret false'

            2
            4
            6
            8
            10

foldl/foldr


            $ list a b c d | foldl lambda acc el . 'echo -n $acc-$el'

            a-b-c-d

            $ list '' a b c d | foldr lambda acc el .\
              'if [[ ! -z $acc ]]; then echo -n $acc-$el; else echo -n $el; fi'

            d-c-b-a

            $ seq 1 4 | foldl lambda acc el . 'echo $(($acc + $el))'

            10

foldl/foldr...


            $ seq 1 4 | foldl lambda acc el . 'echo $(mul $(($acc + 1)) $el)'

            64 # 1 + (1 + 1) * 2 + (4 + 1) * 3 + (15 + 1) * 4 = 64

            $ seq 1 4 | foldr lambda acc el . 'echo $(mul $(($acc + 1)) $el)'

            56 # 1 + (1 + 1) * 4 + (8 + 1) * 3 + (27 + 1) * 2 = 56

tup/tupl/tupr


            $ tup a 1
            (a,1)
            $ tupl $(tup a 1)
            a
            $ tupr $(tup a 1)
            1
            $ tup a 1 | tupl
            a
            $ tup a 1 | tupr
            1

buff


            $ seq 1 10 | buff lambda a b . 'echo $(($a + $b))'
            3
            7
            11
            15
            19
            $ seq 1 10 | buff lambda a b c d e . \
                'echo $(($a + $b + $c + $d + $e))'
            15
            40

zip


            $ list a b c d e f | zip $(seq 1 10)
            (a,1)
            (b,2)
            (c,3)
            (d,4)
            (e,5)
            (f,6)
            $ list a b c d e f | zip $(seq 1 10) | last | tupr
            6

curry


            $ arg='[key1=value1,key2=value2,key3=value3]'
            $ get() {
                local pidx=$1
                local idx=$2
                local arg=$3
                echo $arg | tr -d '[]' | cut -d',' -f$idx | cut -d'=' -f$pidx
            }
            $ curry get_key get 1
            $ curry get_value get 2
            $ get_key 1 $arg
            $ get_value 1 $arg
            key1
            value1

curry...


            $ arg='[key1=value1,key2=value2,key3=value3]'

            seq 1 3 | map lambda a . \
                'tup $(get_key $a $arg) $(get_value $a $arg)'

            (key1,value1)
            (key2,value2)
            (key3,value3)

try


            $ echo 'ls /home' | try λ cmd status ret . \
                'echo $cmd [$status]; echo $ret'

            ls /home [0]
            lost+found#sof-tech#ssledz

            $ echo '/home' | try λ cmd status ret . \
                'echo $cmd [$status]; echo $ret'

            /home [126]
            ../src/fun.sh:#linia#173:#/home:#Jest#katalogiem

scanl


            $ seq 1 5 | scanl lambda a b . 'echo $(($a + $b))'
            1
            3
            6
            10
            15
            $ seq 1 5 | scanl lambda a b . 'echo $(($a + $b))' | last
            15

fun-sh core concept


lambda() {
  lam() {
    local arg
    while [[ $# -gt 0 ]]; do
      arg="$1"; shift
      if [[ $arg = '.' ]]; then
        echo "$@"
        return
      else
        echo "read $arg;"
      fi
    done
  }
  eval $(lam "$@")
}

Examples

counting lines
counting words

Counting lines


            $ curl -Ls https://goo.gl/tE1JUh | wc -l

            254


            $ curl -s curl -Ls https://goo.gl/tE1JUh \
                | zip $(seq 1 1000) | last | tupl

            254


            $ echo 0 | cat - <(curl -Ls https://goo.gl/tE1JUh) \
                | foldl lambda acc el . 'echo $(($acc + 1))'

            254

Counting words


            $ cat <(\
                while read line; do \
                    for word in ${line[@]}; do \
                        echo $word; \
                    done; \
                done \
            < <(curl -Ls https://goo.gl/tE1JUh)) | wc -l

            568


                $ curl -Ls https://goo.gl/tE1JUh | wc -w

                571


            $ echo 0 | cat - <(curl -Ls https://goo.gl/tE1JUh) \
                | map lambda a . 'list $a' \
                | foldl lambda acc el . 'echo $(($acc + 1))'

            568

Resources

Thank you -:)