Concurnas vs Java - Part 1

LinkedIn
Reddit
Hacker News
WhatsApp

Launched in late 2019 Concurnas is an open source Java Virtual Machine (JVM) programming language designed for building reliable, scalable, high performance concurrent, distributed and parallel systems. It presents a progressive, dead easy model of Concurrent computing, offers first class GPU computing support and many more features suited for modern software systems engineering.

Java, released in 1996, has become one of the world's most popular programming languages. This is for a multitude of reasons including its "write once run anywhere" philosophy, its standard library and in its separation as a language from its runtime, the JVM. This brilliant act of separation has enabled third party languages such as Scala, Kotlin and now Concurnas to leverage the capabilities of the JVM all whilst maintaining, for the most part, full compatibility with Java and other JVM languages.

In this three part series (part 2, part 3) we compare and contrast the key features of Concurnas against Java and show why you too will benefit from using Concurnas for your next JVM project!

So let us begin!

Concurrent Programming

One of the strongest points of Concurnas is in its unique concurrency model. Concurnas presents 'isolates' as a means of performing concurrent computing. All code in Concurnas is executed within isolates. Isolates are like threads in Java but lightweight. They are multiplexed on to a number of hardware threads (the number of which is determined by the specification of the machine). From a syntax perspective, since making concurrent programming accessible to everyone has always been a core design goal of Concurnas it is very easy to create and work with isolates. Isolates will copy all mutable dependent data into themselves, thus preventing accidental sharing of state and non deterministic behavior. In this way we are able to write code in a largely sequential manner within isolates, safe in the knowledge that regardless of the underlying concurrent computing capacity of the hardware we are executing our code on, it will behave in the same way. We can spawn as many isolates as we have memory available on our machine, furthermore, from a performance perspective the approach presented in Concurnas is highly performant as isolates do not require switching into the operating system kernel upon context switch.

Concurnas provides 'refs' as a means of sharing concurrently changeable non deterministic state between isolates. Refs are built into the type system of Concurnas, when an isolate encounters a ref which has no value set upon it, it will block until a value has been set. Ref's have the unique property that they may be watched for changes. This enables the reactive programming paradigm to be supported within Concurnas.

Concurnas also embraces the actor model of concurrent computation, which is tremendously useful for providing lightweight services. Furthermore, the actor support within Concurnas supports turning any conventional class (in any JVM language) into an actor.

Isolates and Refs

Here is a simple isolate, the bang ! operator is used to create an isolate:

{System.out.println("Hello world!")}! //! creates an isolate

The above will perform some I/O, printing out to console Hello world! concurrently.

Let's now create an isolate which returns a ref:

msg = {"Hello world!"}! //! creates an isolate, which creates a String and returns it in a ref

System.out.println(msg)//and print it out

The System.out.println call will halt until the value of the ref msg has been set by our spawned isolate. An alternative way to achieve this halting behavior is to use the await keyword:

//msg is a ref...
await(msg)
//...now carry on with the rest of the execution

Whilst an isolate is awaiting a value to be set (either implicitly or explicitly) as par above the underlying worker thread upon which its been multiplexed can context switch to execute other isolates.

Isolate state

Lets now create an isolate to produce a list of numbers:

class CounterHolder(public -n int)

counter = CounterHolder(10)

countDownStr := {
    ret = StringBuilder()
    while(counter.n-- > 0){//we are changing the value of n
        ret.append(counter.n)
    }
    ret.toString()//return a String
}!

System.out.println("countdown from: {counter.n} == {countDownStr}")
//outputs: countdown from: 10 == 9876543210:

Notice how above we are changing the value of n in the counter instance object of type CounterHolder within the isolate. Since this object is mutable it is implicitly copied into the isolate, changes to its contents made by the isolate are visible within the isolate only, thus our output string is able to show the correct initial value when we later refer to it.

Achieving the equivalent in Java requires a lot more code:

public class ConcurrentCounter {
    public static class CounterHolder{
        public int n;
        public CounterHolder(int n) {
            this.n = n;
        }
    }

    public static class IsolateEmulation implements Runnable{
        String returnValue;
        CounterHolder counter;

        public IsolateEmulation(CounterHolder counter) {
            this.counter = counter;
        }

        public void run() {
            StringBuilder sb = new StringBuilder();
            while(counter.n-- > 0) {
                sb.append(counter.n);
            }
            this.returnValue = sb.toString();
        }
    }

    public static void main(String[] args) throws InterruptedException{
        CounterHolder counter = new CounterHolder(10);
        //manually copy state into IsolateEmulation:
        IsolateEmulation isoEmu = new IsolateEmulation(new CounterHolder(counter.n));

        Thread runme = new Thread( isoEmu);
        runme.start();

        runme.join();
        System.out.println("countdown from: "+counter.n+" == " + isoEmu.returnValue);
    }
}

Even with the above implementation in Java there are still some non trivial problems to solve. 1). the thread model scales only to a limited extent, 2). it's very easy to accidentally share state between threads which can lead to non deterministic effects, you can see here that we have had to explicitly copy the value of counter. Both of these points make scaling and testing our solution a challenge.

Reactive Computing

One of the awesome things about refs in Concurnas is that they can be watched for changes and other isolates may react to those changes triggering code as appropriate. This is known as reactive computing. Here is an example where we can calculate a stream of resultant values:

x int:
a int:

y int: = every(x, a){
    x**2 + a
}

x = 4
a = 2

assert y == 18

The every statement above will trigger the body of code whenever its input refs, x and/or a change, this code will itself then return a value which is set to the ref y. y now contains a stream of results of the x**2 + a equation.

We can extend the above with a every statement hooked on to y to print out the stream of values set to it:

every(y){
    System.out.println("{y}")
}

Concurnas provides us with an even more compact form of this expression, we can shorten this every to the following:

y <= x**2 + a

Whilst it is technically possible to implement the above in Java, it would require so much code to implement it to the same level that we'd end up running out of space to discuss anything else in this article! Furthermore, even if we were to do this, we'd only be solving the problem for this one instance, the fundamental problem would still persist as the Java model of concurrency is much more verbose than that presented in Concurnas.

Distributed Computing

The isolate pattern using the bang ! operator may be used in order to perform distributed computing:

//A remote server:
remServer = new com.concurnas.lang.dist.RemoteServer(port = 42001)
remServer.startServer()

//A client:
rm = Remote('localhost', port = 42001)
//execute code remotely, returning a ref
ans int: = {10+10}!(rm.onfailRetry())

//ans == 20

Concurnas will perform automatic data and code dependency distribution to the site of remote computation. Thus we can define classes, data structures and instances of those classes which are totally alien to the RemoteServer which we have connected to, it will receive all the dependent code required to perform the execution. Achieving something similar in Java, even using many open source libraries to assist in this, is very challenging.

Actors

Actors present a neat way in which we can write lightweight services by implementing them like classes, and without explicit consideration for concurrency control.

actor CounterService{
    -n = 0
    def inc(){
        ++n
    }

    def dec(){
        --n
    }

    def get() => n
}

Actors permit only one stream of concurrent execution to mutate their state at a time. Actors, by virtue of their implicit concurrency control, are another special type the instance objects of which may be used within isolates without copying required. For example:

cs = CounterService()

sync{//sync ensures all spawned child isolates complete execution...
    {cs.inc()}!
    {cs.dec()}!
    {cs.inc()}!
}//the above 3 calls occur concurrently and non deterministically

result = cs.get()

There is no scope for inconsistent state inside the CounterService actor.

Actors may also be created of existing classes. For instance, if we wished to create an ArrayList as a service that can be shared between and used by many isolates. We can use the actor keyword in place of the new keyword:

myARService = actor java.util.ArrayList<int>()

Actors are not supported within Java, the best alternative is to use frameworks such as Akka.

parfor

Parfor is a nice little concurrency shortcut which can be used when one is performing a parallel operation on the CPU. For instance, to apply an equation to each element of a list:

inputs = [1, 2, 3, 4, 5, 4, 3, 2, 3, 4]

result = parfor(inp in inputs){
    inp**2 + 3
}
//result ==> [4:, 7:, 12:, 19:, 28:, 19:, 12:, 7:, 12:, 19:]

Again, as with reactive computing whilst it is technically possible to implement the above parfor example in Java, it would require so much code to implement it to the same level that we'd end up running out of space to discuss anything else in this article!

GPU computing

Java does not have native support for GPU computing though there are a number of third party libraries which one can use in order to perform GPU computing including TornadoVM and jocl (which Concurnas leverages behind the scenes).

Concurnas on the other hand has first class citizen support for GPU computing. We can write normal looking, idiomatic Concurnas code and have that code compiled along with the rest of our Concurnas code seamlessly for subsequent execution upon as many GPU's as we have access. Writing the following sort of code, for matrix multiplication using local memory on the GPU is quite straightforward:

val CacheSize = 16

gpukernel 2 matMultLocal(M int, N int, K int, constant A float[2], constant B float[2], global out C float[2]) {
    row = get_local_id(0)
    col = get_local_id(1)
    globalRow = CacheSize*get_group_id(0) + row //row of C (0..M)
    globalCol = CacheSize*get_group_id(1) + col //col of C (0..N)

    //local memory holding cache of CacheSize*CacheSize elements from A and B
    local cacheA = float[CacheSize, CacheSize]
    local cacheb = float[CacheSize, CacheSize]

    acc = 0.0f

    //loop over all tiles
    cacheSize int = K/CacheSize
    for (t=0; t<cacheSize; t++) {
        //cache a section of A and B from global memory into local memory 
        tiledRow = CacheSize*t + row
        tiledCol = CacheSize*t + col
        cacheA[col][row] = A[tiledCol*M + globalRow]
        cacheb[col][row] = B[globalCol*K + tiledRow]

        barrier(true)//ensure all work items finished caching

        for (k=0; k<CacheSize; k++) {//accumulate result for matrix subsections
            acc += cacheA[k][row] * cacheb[col][k]
        }

        barrier(true)//ensure all work items finished before moving on to next cache section
    }

    C[globalCol*M + globalRow] = acc
}

Imperative Programming

Concurnas offers a number of innovations in the area of classical, imperative programming.

Type inference

Concurnas is largely an optionally typed language. This is enabled through its support of type inference. Java has recently added this functionality, in Java 10, for local variables as follows:

var myvar = 12;

Concurnas has a slightly more concise implementation of this, which also works for when defining arrays, matrices and return types of functions:

myvar = 12

myar = [1 2 3 4 5.0]//a double array
mymat = [1 2 3 ; 4 5 6]//an integer matrix

def doMath(upon int) => upon * 2 + 9//inferred as returning an integer

Usage based generic type inference

In Java we are obliged to qualify the generic types of generic instance objects at declaration time:

public static class Holder<X>{
    private java.util.ArrayList<X> li = new  java.util.ArrayList<X>();

    public void add(X x) {
        li.add(x);
    }
}

Holder<Integer> hh = new Holder<Integer>();
hh.add(12);
hh.add(33);

With Concurnas we are able to omit this and have the generic types inferred based on localized usage of said object:

class Holder<X>{
    li = list<X>()

    def add(x X) void {
        li.add(x)
    }
}

hh = new Holder()//infered as being Holder<int>
hh.add(12)
hh.add(33)

Everything returns

In Concurnas all blocks can return values. This also means that the return keyword can usually be omitted from function definitions. Through this feature it is easy to write concise code such as the following:

def choice(a int){//returns a String
    if(a mod 2 == 0){
        "div by two"
    }else{
        "not div by two"
    }
}

result = {
    a = 21
    for(n in 0 to 10){
        a += n
    }
    a//implicit return
}

Conditional Operator

Concurnas features a more intuitive conditional operator. In Java this looks like the following:

boolean result = something() > 10 ? "case one" : "case two";

The equivalent in Concurnas is:

result = "case one" if something() > 10 else "case two"

Looping structures

Concurnas has support for the same looping structures as Java and more! For instance, in Java if we want to write an infinite loop we must write:

while(true){
    doSomething();
}

But in Concurnas a dedicated loop statement is provided:

loop{
    doSomething()
}

Handling an empty list

Concurnas offers a handy solution to the following problem often seen in Java:

java.util.ArrayList<Integer> myList = ...

if(myList.isEmpty()){
    callFunction(0);
}else{
    for(n in myList){
        callFunction(n);
    }
}

In Concurnas we can use an else block to attach a condition to cover the case where we wish to handle an empty list:

myList = ...

for(n in myList){
    callFunction(n)
}else{
    callFunction(0)
}

Loops can return

Since all blocks are able to return values in Concurnas, we're able to write some elegant code. For instance in Java where we would normally write:

int[] myarray = new int[]{1, 2, 3, 4, 5};

java.util.List<Integer> myList = new java.util.ArrayList<Integer>();

for(int n : myarray){
    if(n == 2){
        myList.append(2);
        continue;
    }else if(n == 3){
        continue;
    }else if(n == 9){
        myList.append(-1);
        break;//terminate early
    }
    myList.append(n*10);
}//myList ==> [10, 2, 40, 50]

In Concurnas we can write:

myarray = [1 2 3 4 5]

myList = for(n in myarray){
    if(n == 2){
        continue 2;//add 2 to list and carry on
    }else if(n == 3){
        continue;
    }else if(n == 9){
        break -1;//terminate early
    }
    n * 10
}//myList ==> [10, 2, 40, 50]

Loops with index values

Sometimes it's nice to use iterator style for loops, but also have an index counter like with a c style for loop. With Java we are obliged to use the following pattern (or a c style for loop):

int[] myarray = new int[]{1, 2, 3, 4, 5};

int idx=0;
for(int n : myarray){
    doSomething(n, idx);
    doSomethingElse(n, idx);
    idx++;
}

Whereas in Concurnas this can be achieved with an index variable as follows:

myarray = [1 2 3 4 5]

for(int n : myarray; idx){
    doSomething(n, idx);
    doSomethingElse(n, idx);
}

toBoolean

Objects in both Java and Concurnas have a toString method, in Concurnas all objects also have an additional toBoolean method. This means that the following code in Java:

class Holder<X>{
    private X x;
    public void setX(X x) {
        this.x = x;
    }

    public boolean toBoolean() {
        return this.x != null;
    }
}

Holder<Integer> holder = new Holder<Integer>();

if(holder.toBoolean()) {
    //x has been set inside holder!
}

May be simplified in Concurnas as:

class Holder<X>{
    +x X?
    override toBoolean() => this.x <> null
}

holder = new Holder<Integer>()

if(holder) {//equivalent to if(holder.toBoolean()){...
    //x has been set inside holder!
}

Furthermore, if the type of the expression being checked in the if statement is nullable, it shall first be examined to see if it's null. Thus the following is valid:

holder Holder<Integer>? = //...

if(holder) {//equivalent to: holder <> null and holder.toBoolean()
    //x has been set inside holder!
}

toBoolean for lists

Lists represent a special case in the aforementioned toBoolean logic. For lists, maps and sets toBoolean is mapped to not _.isEmpty(). Thus the equivalent of the following Java code:

java.util.List<Integer> myList = new java.util.ArrayList<Integer>();
//...
if(!myList.isEmpty()){
    //do something with non empty list
}

Is the following in Concurnas:

myList = list<int>()
//...
if(myList){
    //do something with non empty list
}

Functions

Functions and methods in Concurnas can be an optionally concise. With Java we are obliged to stick to one verbose way of doing things:

public static int doMath(int upon) {
    return upon * 2 + 9;        
}

Whereas in Concurnas we may be optionally concise. The following are functionally identical:

def doMath(upon int) int{//most verbose form
    return upon * 2 + 9
}

def doMath(upon int){//inference of return type
    return upon * 2 + 9
}

def doMath(upon int){
    upon * 2 + 9//implicit returns
}

def doMath(upon int) => upon * 2 + 9//most compact definition

Generic Functions

Like Java, Concurnas supports generic functions, here is a Java example:

public static <Gen> java.util.ArrayList<Gen> makeList(Gen g) {
    java.util.ArrayList<Gen> ret = new  java.util.ArrayList<Gen>();
    ret.add(g);
    return ret;
}

The equivalent in Concurnas is slightly more intuitive in its positioning of the generic type qualification:

def makeList<Gen>(g Gen) => list<Gen>()..add(g)

Nested functions

Concurnas has support for nested functions. They are a nice option for avoiding code duplication, they also make it easier to read code that uses companion functions (i.e. whose utility is restricted to assisting only the function within which they are nested):

def pairProcessor(items list<int>){
    plusWhat = 10
    def op(x int) => x*2 + plusWhat//nested function

    for(n in items){
        op(n), op(n+1)
    }
}

//a call: pairProcessor([1, 2, 3]) returns: [(12, 14), (14, 16), (16, 18)]

Java does not support this functionality.

Default arguments

Concurnas has support for default arguments. Consider the following typical case of Java code:

public static int doSomething(a int){
    return a + 100;
}

public static int doSomething(){
    return doSomething(12);
}

The equivalent in Concurnas is much more compact:

def doSomething(a = 12) => a + 100

The above Concurnas code may be invoked as follows:

doSomething()
doSomething(99)
//etc...

Named arguments

Named arguments are not a feature of Java. With Concurnas the following is possible:

def doSomething(a = 12, b = 100) => a + b

doSomething(a=78)//equivalent to: doSomething(78, 100)

Java does not support this.

What's next?

Checkout the next article (part 2) in the series for more differences between Concurnas and Java.