Concurnas is an open source JVM programming language designed for building reliable, scalable, high performance concurrent, distributed and parallel systems. Launched in late 2019 is an open source Java Virtual Machine (JVM) programming language designed for building reliable, scalable, high performance concurrent, distributed and parallel systems. It presents a progressive, dead easy model of Concurrent computing, offers first class GPU computing support and many more features suited for modern software systems engineering.
The challenges faced by the modern enterprise require software solutions that are both highly performant, scalable and quick to market. Traditionally, statically typed languages (such as Java have dominated the high performance computing market, whereas dynamically typed languages Python have been favored by domain experts working in areas such as scientific computing, research and development, prototyping and rapid application development. Statically typed languages are harder than dynamic languages for non technical users to work with but generally have superior performance. And so it has been that we've had to make a choice between between runtime performance and developer performance, statically typed and dynamically typed languages. With Concurnas this is different, with Concurnas we are able to obtain both the advantages of statically typed languages,and those of dynamically typed languages. In this way we no longer need to sacrifice developer performance and time to market for performance, we win on both counts: we get Python like syntax with Java like speed!
Concurnas makes use of type inference in order to present an easy to learn dynamically typed like syntax. At its core it is a statically typed language that is type safe and which runs upon the Java Virtual Machine (JVM). It is through this implementation upon the JVM that Concurnas is afforded:
Furthermore, since Concurnas has the performance of a statically typed language, with the ease of use of a dynamically typed one, and is designed to scale, teams are able to use both the same programming language: Concurnas and code written in Concurnas as their core programming language through all stages of software development: from research, prototyping, development, production and scaling. There is no need to switch between different languages in teams made up of heterogeneous domain experts and software engineers, one language can be used - reducing cost and time to market.
From a design perspective Concurnas has been designed as a language which:
Concurnas is able to boost productivity by virtue of the fact that its concise syntax and first class citizen support for common enterprise computing problems enable us to be productive by writing fewer lines of code. This reduces the changes of bugs, increases productivity and helps us get to market with solutions quicker.
Concurnas is a multi-paradigm programming language which aims to unify the best features of the Imperative, Functional, Object Oriented and emerging Reactive programming paradigms.
Almost all programmers are familiar with imperative programming constructs such as variable assignment, control flow statements (if-elif-else), for loops, assertions, exceptions etc. Concurnas deliberately offers no surprises here, what we learned about programming in school works as expected. Concurnas does however offer a number of useful features in the imperative space which makes it very concise and easy to use:
The Functional programming paradigm represents an often untapped resource of really good ways of making programs easier to write, read and reason about. Concurnas makes use of a number of these features:
The object oriented programming paradigm has become synonymous with enterprise software development, for many debatably good and bad reasons. What's for sure though is that traditional programming languages often place the burden of implementation of object orientation upon the developer. In practice this results in a tremendous amount of boiler plate code. With Concurnas this is different. Concurnas takes the hard work and repetition out of writing object oriented software whilst embracing modern software engineering best practices. We have support for the ordinary established features of object orientation including: classes (concrete, abstract, inheritance), encapsulation, method overloading, dynamic binding, generics (in out, wild cards, bounded), enumerations and annotations. Concurnas also has support for a number of emerging features/best practices including:
The isolate and ref oriented model of Concurrency in Concurnas (explored later) naturally lends itself to supporting the emerging reactive programming paradigm. In reactive programming we can create implicit dependency pipelines of data throughout our code such that changes in data are automatically propagated throughout our system with our code written in such a way that it reacts to those changes, maybe itself further propagating change.
The reactive style of programming tends to be very intuitive for solving many types of algorithmic problem, particularly those involving asynchronous flows of concurrent data. Unlike traditional imperative programming where one is pulling data into a location for further processing, with the reactive model data is effectively pushed into a location for processing.
Expressing reactivity in Concurnas is usually doable in only a few lines of code.
One of the most challenging aspects of modern enterprise programming, along with the likes of Artificial intelligence/Machine learning, computer graphics and security, is concurrent programming. We want Concurrent programming to be accessible to everyone, not just software engineers. Concurnas was originally designed from its core as a concurrent programming language. The Concurnas concurrency model is thread, critical section and lock free, which of course naturally precludes deadlocks. The model of concurrency computation offered by Concurnas generally results in us writing software which looks single threaded by virtue of the fact that the hard work of managing concurrent state is abstracted away.
For better or worse we exist in an age of multi-core computation, thus for our programs to be as performant as possible they need to be able to take advantage of the multiple cores available to them. Code written to be concurrent in Concurnas is designed to behave in the same way whether it is run on a single core or multi-core machine, automatically scaling to take advantage of the available hardware.
Instead of sharing memory, messages are used to communicate changes in state, this greatly simplifies the programming model of concurrency, whilst preventing non deterministic changes in state which can otherwise occur in traditional shared memory languages when that shared memory is not properly managed. This is achieved through the use of what we term: isolates and ref's. Isolates are the underlying unit of concurrent computing in Concurnas, ref's are a means of communicating changes in state in a controlled manner.
All code in Concurnas is executed within isolates. Isolates are mapped onto the underlying hardware threads of the machine upon which one is running one's code. Isolates operate within their own dedicated memory space. All state used in an isolate is copied with the exception of references, actors and some specially marked shareable classes. This isolation of state makes reasoning and designing concurrent algorithms easier since it reduces the need to code for synchronization and race conditions.
Refs allow us to share state between isolates in a controlled manner. They are not copied between isolates, rather they are shared but provide controlled, atomic access to their changeable state. If a value is not yet set on a ref when it is requested, execution is paused and other isolates are allowed to execute until a value has been set, after which execution continues. Ref's take an optimistic approach to concurrency control which shields the end user from managing locks and critical sections.
As has already been mentioned, the isolate and ref model of concurrency in Concurnas naturally lends itself to supporting the emerging paradigm of reactive programming. Elements of reactivity (onchange and every blocks) are triggered for execution upon changes to their input ref's and remain paused otherwise. This approach of suspending and resuming execution is achieved by virtue of the fact that isolates are continuations - an executing isolate may save its state and yield its underling multiplexed thread to another pending isolate thus enabling cooperative multitasking.
We are able to perform transactional computing with refs. Concurnas supports software transactional memory via the trans block keyword. This allows us to make changes to one or more refs and have those changes visible outside of our transaction on an atomic basis.
Concurnas offers first class citizen support for Actors. The Actor model of computation has become popular in recent years as multi-core chips have become ubiquitous. Actors in Concurnas enable us to build micro-services. Each actor operates within its own dedicated isolate. Actors look and behave just like regular objects from the perspective of the calling code.
Actors allow us to write service code in a single threaded style and be able to run them safety in a concurrent manner without having to concern ourselves with synchronization or race conditions as each call to an actor runs sequentially and has exclusive access to the sate of the actor until it completes. Naturally, because actors implement their own concurrency control they are safe to be shared between multiple isolates without requiring copying.
Concurnas offers a succinct syntax for defining and working with actors.
A common pattern in concurrent systems engineering is to want to have some for of time based trigger, "wait 10 seconds then do x" or "do y every 3 seconds" etc or "perform this action at this certain time". At Concurnas we refer to this as temporal computing.
Temporal computing is supported via the built in Pulsar library found at: com.concurnas.lang.pulsar. This library allows us to schedule activities to take place in the future after a certain amount of time has elapsed. It also allows us to schedule tasks for repetition.
Concurnas offers both a realtime implementation of pulsars and a 'Frozen'-time implementation (designed primarily for testing temporal applications), which permits time to be treated as a variable to be injected.
Concurnas has first class citizen support for parallel for loops. These are a convenient and intuitive mechanism for performing task based operations in the context of a loop, in parallel. They have the same syntax as conventional loops in Concurnas.
Concurnas provides an easy to use, intuitive and non invasive solution for distributed computing which acts as an extension of the isolate model of concurrent computation. In this way we are able to use the concurrent computing model which we are used to/have already built our software using and can easily extend this to working on a remote basis. In this way as our computational requirements grow we can easily scale our solutions to accommodate.
Concurnas abstracts away much of the hard work Concerning managing distributed computing, allowing us to focus on the more interesting problems which we are trying to solve. For instance, with Concurnas remote servers will cache results and wait for clients to reconnect in the event of network failure.
With Concurnas, remote servers are able to be configured such that they can run any code submit to them deemed safe for execution.
The internal Concurnas distributed computing API includes a provision for dependency resolution which itself is quite sophisticated and in the interests of performance is able to perform static code dependency analysis in order to determine and preemptively provide upfront the code that is required in order to execute a request.
In this way clients are able to submit any defined code and have this run on a remote machine with dependent code automatically distributed to the remote machine. This greatly simplifies the usually arduous process of remote code dependency distribution.
Dependant object state is also distributed for use with remote execution, rendering the execution semantic between local and remote computation identical. This makes it very easy for us to start building a solution locally, and to then later scale that solution up on a distributed platform.
In languages such as Java there exist a number of commercial products for performing this form of off heap memory management, often costing thousands of dollars per user per year. With Concurnas you get this for free as an integral part of the language!
Concurnas has support for off heap memory in the form of stores and maps. These allow us to work with datasets which are far greater than that which can be held within the managed garbage collected portion of memory used by Concurnas termed the heap. Additionally by using off heap memory we gain a tremendous amount of control over how we manage memory and can reserve portions of memory separate from garbage collection which is incredibly useful for high performance computing.
Objects are stored off heap in binary format. With Concurnas all objects are quickly serializable and deserializable to and from this format by default.
Concurnas presents two variants of off heap storage, for RAM and disk - for use contingent on the amount of data one is working with. The off heap disk variant is backed by memory mapped files which greatly enhances performance as spatially localized data is cached in memory.
The Key-Value pair map has become an industry standard too for persisting data. With Concurnas support is provided in the form of off heap maps. Both a RAM and disk backed variant of these maps is provided.
The off heap map disk variant is backed by memory mapped files which greatly enhances performance as spatially localized data is cached in memory. Additionally, the disk based variant is capable of long term persistence of data.
One of the strongest points of the Concurnas off heap map implementation is its support for schema evolution. We term schema evolution as changes to a classes fields after it has been persisted.
Schema evolution turns out to be a surprisingly normal operation performed in enterprise computing. Traditionally this would cause a problem for us upon deserialization since the persisted version of the class code would not match that of the current 'live' version, but Concurnas is largely able to account for these sorts of evolutionary changes including:
Concurnas is able to account for the above changes to object schemas when serializing and deserializing data from persistent storage. This saves us the headache of explicit migration.
Concurnas has built in first class citizen support for programming GPUs and associated parallel computing constructs. GPU's are massively data parallel computation devices which are perfect for solving SIMD (single instruction, multiple data) and normally CPU bound problems. Compared to a single CPU core algorithm implementation, in solving a computation bound problem, it is common to be able to obtain up to a 100x speed improvement (two orders of magnitude) when using a GPU! And this is with a far reduced energy and hardware cost per gigaflop relative to CPU based computation. All modern graphics cards have a built in GPU and there exist dedicated GPU computation devices. In fact some of the world's leading supercomputers achieve their parallelization through the use of dedicated GPU hardware. With Concurnas this power is your as well.
Support is provided via interfacing to OpenCL - an excellent multi-platform API for GPU computing. OpenCL is supported by the big three GPU manufacturers - NVidia, AMD and Intel. However, even with conventional, raw OpenCL (or any GPU computing platform for that matter) one must write a lot of boilerplate code and perform a lot of non work related management in order to get computation done on the GPU. A such, authoring large applications can be an intimidating prospect. With Concurnas you will see that this boilerplate code is minimized, allowing the developer to focus on solving real business problems. Additionally there is no need to learn (and have to paradigm shift into) C or C++ which is the native language used by OpenCL for expressing parallel functions on the GPU as functions marked as parallel in Concurnas are automatically translated into a form understandable by the GPU/OpenCL. GPU computing is for everyone.
Modern enterprise computing involves working with data organized into data structures. Some of the most common data structures used are arrays, matrices, lists, sets, maps and numerical ranges.
Concurnas has advanced first class citizen support for not just defining arrays, matrices and n-dimensional arrays but working with them to. Concatenation operators are provided to aid in joining of n dimensional arrays and matrices as well as various means to extract values.
In addition to n-dimensional arrays, Concurnas supports lists, sets, maps and ranges. Concurnas has built in first class citizen support for these essential data structures. With Concurnas you can define lists, maps and ranges in place with convenient, dedicated Python like compact syntax. Concurnas supports the most commonly used operations on these data structures directly, such as adding, removing and checking for the presence of elements.
Concurnas leverages the Java JDK implementation of these data structures. They make use of generics in order to support convenient multi typing of structures. Furthermore, Concurnas employs usage based type inference in order to infer what these generic types should be qualified as at compilation time based on the usage of the data structure.
Concurnas, like other modern programming languages such as Kotlin and Swift has null safety built into its type system. Concurnas requires that a variable's type be explicitly declared as being nullable in order for it to be assigned null, or potentially null value. When working with nullable types, Concurnas enforces a number of restrictions so as to avoid the risk of throwing a NullPointerException. In this way Concurnas enables us to write safe code which is guaranteed to be largely free of NullPointerException's (an unexpected NullPointerException being widely recognized as being one of the most common bugs in modern software engineering).
Concurnas includes sophisticated logic for working with nullable types to cater for situations in which they have already been established as being not null. This makes working with nullable types easy.
Concurnas includes three useful built in operators that can be used when working with nullable types:
Concurnas offers first class citizen support for dependency injection through Object Providers. Dependency Injection (DI) is a modern software engineering technique for building object oriented systems. The core principle of DI is to separate behavior from dependency resolution (the "plumbing"). In doing so one is obliged to compartmentalize one's software into core functionality, making dependent behavior injectable and thus separating that injectable code from the "plumbing". This has benefits of aiding:
For the above reasons and more it's clear why DI has become such a useful tool for modern software development. Concurnas object providers build upon the successes of the likes of Spring and Google Guice. Concurnas object providers offer a non invasive and concise, mechanism by which code may be both marked for dependency injection (via the inject keyword), configured (via object provider's) and provided for further execution with all dependencies (including transitive) resolved. Best of all, this all happens in idiomatic Concurnas code and is validated at compile time!
Concurnas has first class citizen support for language extensions.
Language extensions are an incredibly powerful feature of Concurnas which enable guest code to be written in languages other than Concurnas. Language extensions reduce the amount of code one would otherwise be required to write and reduce the distance between the problem domain one is operating in, and Concurnas, by allowing us to use the most appropriate language for the task.
Concurnas language extensions operate entirely at compilation time and so have no negative impact upon performance, furthermore, as with Concurnas in general, compile time checking of code is employed, which greatly reduces the likelihood of an error at runtime. Concurnas Language extensions seamlessly integrate with idiomatic Concurnas code and may be used at any point in a Concurnas program. Language extensions may use and produce any component of Concurnas syntax, such as variables, methods, classes etc
Concurnas Language extensions are easy to create and greatly reduce the amount of work a language implementor must perform in order to bring a language into production. They are required to output valid Concurnas code in the form of a String. This Concurnas code is then processed as par normal Concurnas code in subsequent compilation phases. One need focus only upon the 'front-end' aspects of language development, i.e. tokenization, parsing and semantic analysis (which tend to be where the most value add resides), leaving the hard work on code generation etc to the core Concurnas compiler.
Concurnas Language extensions provide an elegant and concise API for language integration (to access variables in scope, types etc), which supports streamline interrogation of the core Concurnas compiler and a convenient mechanism for reporting errors back to the code author as par normal Concurnas compilation.
Concurnas is a great language for building Domain Specific Languages (DSL's). DSL's enable us to define our own programming languages on top of the functionality offered by the host language in order to achieve the following objectives:
The aspects of Concurnas which make achieving the above objectives possible are as follows:
Concurnas is free and open source, licensed under the popular and permissive MIT license.
Concurnas Ltd. was established in 2018 as a commercial entity in order to support the ongoing development of the Concurnas programming language as open source and to provide commercial support for Concurnas in addition to consulting. For more information on our support packages please see our Support Plans and Consulting Services here.