StateFun.jl - Flink Stateful Functions Julia SDK

July 18, 2021

Setting up Julia


Create a new package

I’ll follow along with the instructions here:

git init
git add -A
git remote add origin
git checkout -b main
git commit -am "initial commit"
git push -u origin main

Then you can tell Julia about your github address:

(@v1.6) pkg> rm StateFun
(@v1.6) pkg> add
(@v1.6) pkg> dev StateFun


(@v1.6) pkg> activate StateFun
(StateFun) pkg> add Revise Test
(StateFun) pkg> test StateFun
     Testing Running tests...
     Testing StateFun tests passed

Building StateFun proto files

In Julia REPL:

]activate; add ProtoBuf  # note: you must have ProtoBuf installed in the base environment for this to work.
]activate StateFun


We’re going to borrow the python version of the greeter example in flink-statefun-playground, except we’ll swap out the python service for a julia service.

Python version

First get the python version working:

Julia Statefun Functions

The Julia server

We’ll make a very simple http server using Mux.jl, and save it as scripts/run_server.jl:

using Mux, StateFun, ProtoBuf

@app app = (Mux.defaults, # use Mux.prod_defaults later
    page("/statefun", req -> begin
        # ...

# ...
wait(serve(app, port))

This server will listen at http://localhost:8000/statefun and handle anything that comes in. This is the same address that the python greeter example was listening.

Wrap our Julia server in a Docker image

This is the Dockerfile which will run our Julia server:

FROM julia:1.6.2

WORKDIR /opt/StateFun

# copy this package into the /opt directory in the container
# ensure this is a git repo... Pkg needs it when adding
RUN apt-get -y update && apt-get -y install git
RUN git init

# warm up
COPY Project.toml .
COPY Manifest.toml .
RUN julia -e 'using Pkg; Pkg.activate("."); Pkg.precompile()'

# add the package and precompile it
COPY src src
COPY scripts/run_server.jl .

# ensure the package's environment will be activated on julia startup
RUN mkdir -p /root/.julia/config
RUN echo 'using Pkg; Pkg.activate(".")' > /root/.julia/config/startup.jl

# run julia
ENTRYPOINT ["julia", "run_server.jl"]

You should then build it:

docker build -t statefunjl/greeter_example .

Swap out python for julia service

In the docker-compose.yml file, replace the functions block with this:

    image: statefunjl/greeter_example
      - "8000"

This is going to launch the julia docker container we made instead of a python version.

Run it all


cd /path/to/flink-statefun-playground/python/greeter

Now we’re going to start up the docker-compose cluster which will contain:

docker-compose up -d

After a few seconds, the containers should be running and your Julia handler should be logging what it sees:

docker-compose logs -f functions