Introduction to EK9

EK9 is a modern general purpose programming language that fuses Object Oriented and Functional programming techniques with controlled Generic/ Aspect Oriented Programming (AOP). It pulls a number of widely used design patterns and framework ideas into the language. It provides a modular software ecosystem as part of the compiler /language itself.

It has been designed to be used by very young/inexperienced developers; right through to the 'enterprise' developers. It accomplishes this by providing layered constructs that start simple but scale to cater for sophisticated software solutions.

You may ask; can one single language fuse all those different methodologies together, should they be fused together? Are the compromises that have been made too severe to make any of these techniques truly useful? Mixing what is normally third party libraries and build tools into a single software development platform might be viewed as a backwards step by many.

The 'hello world' section below gives you a quick glimpse of the layout.

Hello, World in EK9

#!ek9
defines module introduction
  defines program
  
    HelloWorld()
      stdout <- Stdout()
      stdout.println("Hello, World")
//EOF

Lets assume this is stored in a file called helloworld.ek9; we would run that program with the following command:

  • $ ek9 helloworld.ek9
  • On Linux/Unix/MacOS you can use this as well/instead
  • $ ./helloworld.ek9

As you can see the layout for EK9 is simple and minimal (Python like in many ways), it uses module to represent a name space (introduction) a construct part (program) and then the program itself (HelloWorld) with its body. The <- is a declaration of a variable of a particular (inferred) type (a class called Stdout which can be used to print out Strings).

Main Features

Why try EK9? Well most of the features in EK9 you'll find in most other modern languages. But there are a few fairly unique capabilities that you may find compelling. It is really the combinational effect of the syntax/layering and structure that is innovative and different.

General Features
Specific new Features

This introduction page is quite very long and full of discussion/rational/logic/licensing if you really just want to get a good look at the language then start with the list of examples or sections on structure, basics and then operators.

You may consider this manual as a stream of consciousness; your criticism is accepted. As this is 'open source' please write a better manual to contribute; it will be gratefully received.

There is a significant amount of information in this manual, do not expect to be able to get through it all in one go. It will take time to understand it all. There are examples throughout, but there are also sections that contain code snips to solve general problems in different ways using EK9. You may find a review of these as helpful as reading these pages.

The examples do actually work with the current prototype compiler, so the outputs shown are the actual outputs from the example programs. So this is not just theoretical ideas/vapourware! But the compiler is incomplete even as a prototype (currently). The first reference implementation of the compiler is now being developed; so join in on its development.

If you are an experienced developer or just want to see what the language is like in more detail; then take a look here for new starters and here for more experienced developers to see what EK9 has to offer you. If on the other hand you've read this and want to understand how mature EK9 is in terms of tooling, compilers, IDE and community the look at the tooling section.

TL;DR

To understand the licensing and current state of development, jump to those sections or just keep reading.

However, if you have just stumbled upon this site and are new to software development and think you might use EK9, there is a brief overview of other languages you might like to consider. This is not intended to dissuade you from trying EK9, but to help you make an informed decision as to which language might suit you.

There is a brief outline of other main-stream current languages that EK9 has taken aspects of design/structure from; each of which are excellent in their own right. Some of the key aspects of development for any developer will be supporting tools (IDE), ease of syntax, a big library of functions/classes/components and critically wide adoption by many developers with code examples and a supportive and informed developer community.

The purpose of creating EK9

EK9 has been designed from the outset to reduce the amount of boiler plate code, punctuation (syntax) and provides pragmatic type inference. It borrows constructs and syntax from other languages (layout, functions, classes, etc); so it should feel quite familiar in many ways, but defines it several new concepts. Furthermore it (re)introduces some older Unix ideas such as streaming/pipelining and limited operator overloading that will be familiar to Unix/Linux and C/C++ developers. If you want to see the main additions over and above other languages then look here.

Like most new languages there is no totally new concept that's going to change the world, but more abstractions and improvements (opinion). The higher level abstractions such as classes and traits could of course all be implemented in languages like 'C'. Functions could have been implemented in 'assembly' code using subroutines. Furthermore classes could be implemented as 'C' structs with a virtual function table for the methods. Traits/Interfaces are nothing more than abstract classes with no properties and only method signatures.

Why create these new constructs, if they could all be defined in other existing ways?

Concepts and abstractions are the value being added (not just ease of implementation). See the Java example below - while it only uses a class; it does tell the developer more about how that class is intended to be used (by using Annotations).

While the much higher level constructs such as components and Inversion of Control could be seen as being excessive in terms of adding more abstractions - the same could be said of functions, classes and traits (if you are or were a C or assembler programmer).

It has been said that 'Library Design is Language Design'. In Java this has been used widely in the form of Annotations and especially by 'Spring' (see example below).

...
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;

@Configuration
@PropertySource (name = "employeeProperties", value = "employee.properties")
@ConfigurationProperties
public class EmployeeConfig {
}
...

As time goes by; it becomes more obvious that a 'design pattern' such as assembly code subroutines could be formalised into a new language and be called a function. C Structs and virtual function pointers could be called a class. Abstract classes could be called a Trait/Interface (enabling more flexibility and decoupling in a polymorphic manner). Classes used in specific contexts could become components. Once you have components you may wish to only allow them to be used in a specific way/context (that is not suitable for just classes).

If you look at the 'Spring' snip above it becomes quite obvious that are are parts of it that could be pulled right into the language itself. Then the compiler can actually do a lot more to ensure that those concepts are only being used in the right way (at compile time - not just run time).

Trying to create a programming language that is easy to read, easy to get started with and that can be used for small, medium, large and very large applications is a challenge. You may view some of the early examples as being trivial and quite easy to accomplish in many other languages. Please take the time to view more of the examples especially in the structure and standard types sections as these start simple; but get progressively more fully featured to show the language capabilities.

Why was it created?

Like most programming languages it fills a perceived gap to some extent. It will be obvious from the 'hello world' example that the Unix/C/Java and Python have been a major influence. It was felt that there were features that could be incorporated from other languages; some very old like Haskell, some new like Swift and some like Java and C# that have proven robust/utilitarian over time. The use of several long lived design patterns and the adoption and acceptance of HTTP/REST/HTML/CSS/SQL have also had a major impact on the design.

Some languages (Java in particular) have struggled to incorporate new ideas in an elegant manner (opinion), others like Python have not maintained backwards compatibility in every release. This is not intended to be a criticism of either language. It is understandable as once the DNA of a language has been established, additions such as Generics and Functions are never going to be as fully incorprated as they could have been if designed in from the outset.

But rather than try to retrofit a number of interesting/powerful concepts into an existing language, experiments (on paper) were tried to see how such diverse and seemingly incompatible concepts could be forged together. It then became obvious that a new language was actually necessary. Some could question the need to forge these ideas together at all - in which case please read the other languages section as there are many good alternatives to EK9.

Specific third party API's and libraries using Annotations can provide some extension to existing languages; but they are all non-standard and not really compiled and checked or checks are delayed until runtime.

The Elephant in the Braces { }

Why was the Python like indentation approach adopted? Rather than the more familiar '{' '}'; as used in C, C++, C#, Java, Javascript, Scala, Dart, Kotlin languages. All of which use this mechanism to define blocks?

For the following reason ';}});'. Excessive/confusing punctuation! While the Python like format does not suit everyone and there are drawbacks to adopting that layout; the clarity of avoiding too much punctuation especially when mixing {, [ and ( in combination is worth it (opinion). White space is invisible - which has both positive and negative repercussions.

Lack of clarity on the eye with excessive punctuation is the main reason EK9 follows a similar route to Python when it comes to layout. Other markup formats like YAML also use this indentation mechanism.

The Current Situation in Development of EK9

At this current point in time EK9 would probably not suit a new developer. Tooling and library support is very limited and the community still has to be formed and grow to the level needed.

But if you are an experienced developer or tester and can see the potential here; please continue to read on and hopefully you will want to get involved and help drive EK9 forward. You may even build significant parts of the tool chain, package manager and standard libraries or really help on testing and documentation.

If you are just looking for something new/interesting/challenging or really want to put your stamp on some opensource code and have it used world wide; then read on and see if this language is something you'd like to use/shape. Maybe you find some of the structure/layout and syntax in other languages frustrating/limiting or possibly you think package management with other languages could be enhanced. Perhaps you expect compiler error messages to be much more useful, now you can decide what the error messages should be.

Creating a modern language via opensource from scratch is a very long term undertaking but needs more than just coders. Testing and documentation are just as critical as the language, compiler and IDE plug-ins. If you feel these documents that explain EK9 can be improved and made clearer then we really want to hear from you; you can update them with your own contributions. With so many good modern programming languages available now; the bar to creating something new and compelling is really very high. Producing something that is only slightly better; means it just won't make the cut and be adopted.

Many languages take decades to establish themselves, some always remain niche. Many however; (like C) will probably be around forever. Hopefully EK9 will be found useful in some form or other for various developers that find it suitable for the particular development they need to undertake.

The creation of something new like this is not aimed at trying to denigrate other languages or frameworks; but is in many ways just another go at trying to evolve programming by adding alternative constructs. It may or may not end up being successful or it may just find a niche, but it is important that evolution is at least attempted.

So come and contribute on github, no matter how small your contribution it will be valued; it will be one more 'brick' in the 'building'.

Current development and tooling

The initial syntax and semantic constructs are complete for EK9 (at least for the first release), there is a prototype grammar for it and also a prototype compiler actually a transpiler (into Java), but there is no reason to assume it has to remain this way. Using LLVM or WebAssembly (Wasm) to create native binaries is quite possibe/desirable.

So all the code examples you see in these web pages, will compile and will execute to produce the results you would expect. But the grammar has not yet been optimised and the compiler/transpiler and code generation phases are still prototypes and not at production level.

But importantly while the prototype compiler generates Java code that is then compiled to produce a runnable application; no decision has yet been taken on whether to create native binaries, wasm, transpile to javascript, use JVM byte code or just continue to generate Java. It could be that MSIL or CIL is produced to run in Microsoft environments. All of these options could be implemented! The EK9 compiler has the concept of a target architecture.

The internal structure of the prototype compiler is such at it creates an IR that can then be used to produce any type of output (even another intermediate form). So if you've always wanted to do the low level code generation such as loop unrolling, register optimisation, built in memory management and all the other things needed in the final stages of compilation to produce a native binary; here is a blank canvas for you but all the colours and brushed are provided.

Timing

So are you wasting your time reading this? Is this document on how to use the language premature?

EK9 has been through many phases of definition and redefinition from 2010 onwards in terms of initial concept (all on paper). So if you're looking for something you can use in anger for production delivery now - EK9 is not ready. But if you want to get in on the ground floor and contribute what you can - then yes this is a very good time to get started.

However it was only in 2018 that the first grammar was actually written and even that has undergone enormous change. But the EK9 language has now found itself. Basically this means that there have been a few decisions taken that constrain, enable, restrict and link programming concepts together (like DNA in a way). This means that at a fundamental level the language has iterated through a number of steps (sometimes retrograde) to find balance.

This might all sound a little peculiar, but now is the time to actually define what the language is (and what it is not), before actually looking to move from prototype to production. During this process many features and capabilities have been tried, removed, reintroduced and in some case removed again; in general this is because they created some type of mismatch or contradiction (even though some of those features were really desirable).

By documenting this now - after prototyping but before the first reference implementation; a concrete target of what is acceptable as an EK9 source file has been defined. This should mean that the documentation/requirements and real examples of EK9 code are created ahead of the first production release (but lets not delude ourselves this is likely to take a decade or more).

In general it has been noted that information/documentation, support, code examples and community play a much more significant role in language adoption than the nature of the language itself. While it is hoped that developers will find the language compelling/elegant/succinct/powerful and fun; it is accepted and understood that it is really communities that create robust applications and drive widespread adoption. With so many good programming languages developed in the last 10 years alone (and more like Ecstasy in development) the language, runtime and supporting tooling are going to have to be very very good.

I have personally benefited throughout my whole working life from opensource code/tools and operating systems.

Having used Pascal, C, C++, ADA, Java, Python, Javascript, KSH, CSH, BASH, Unix, applied many design patterns and used a wide range of API's/Frameworks; I could see clear advantages in one over another in specific areas. But some good ideas had become too complex, ran out of steam, had special cases or forced strange sequences of punctuation that just 'jarred'. This is my attempt to forge several ideas together and contribute something back to opensource.

I'm sure there will be some developers that find the whole approach in EK9 'wrong minded', for some it will be too big, for others the layout and syntax just won't feel right. Bjarne Stroustrup has some comments on languages that are worth reading and there are other languages available if EK9 does not feel right for you.

Licensing

From the outset it needs to be understood that the EK9 language, the reference implementation of the compiler, tests, documents (such as this site) standard libraries and package manager are opensource and that no monitary payment will ever be required to access the source code.

All contributions are of course subject to copyright (for the author making the contribution). So if you contribute to EK9 in some form or other, your contribution is yours and copyright is yours. Clearly with opensource layer upon layer of contributers modify, improve and augment content, so the final state of any source/test/document is not any one single persons copyright, but your changes are.

The EK9 language has trade mark. This is used to determine what constitutes a valid EK9 source code file; as this has to be controlled and managed for compatibility reasons. Services to provide additional resources, documents, training and building stable releases etc can all be commercial and paid for; if there is a market for such activity.

This means that other people/companies will be able to run a commerical business based on the labour and effort of all those that have contributed to this opensource language. But copyright/credits must be retained.

It is hoped that this approach honours the true nature of opensource development, allows for widespread contributions but also enables a viable and reasonable commercial market to grow and provide value for all (while not always in terms of monitary value).

To have wide spread adoption and hence the biggest impact there is a symbiotic relationship between opensource and commercial activities. As mentioned before I've built my whole career on learning languages and applying those skills working/contracting for companies. Commercial activities are not inherently evil or bad, but they have to be controlled and balanced; but neither do we live in a socialist/communist idealised world (counter to human nature).

The MIT license is therefore the right opensource license to adopt.

In short you will always have free access the contributions made to this opensource language with the ability to build it all yourself. EK9 source is made available under the MIT opensource license.

EK9 language

EK9 is a compiled language that is strongly typed (there was not an EK1...EK8 and there won't be an EK10).

New Starters

Below is a summary of EK9, for more details on how the EK9 programming language is structured and how to get started see the section on structure. EK9 has all the normal constructs and concepts of most imperative languages this includes basics for variable declarations and how to put in code comments (and the different types of comments).

Read the section on basic types that are included in EK9 (Integers, Strings, etc) this; in conjunction with the section on operators should enable you to complete some basic processing. EK9 supports many different types of operators - including fuzzy match.

Look at the flow control section to understand how if/else, for/do/while and switch directives work, they are similar to most languages but they have some very powerful additions out-of the-box (guarded assignments and multiple values in switch). The section on exceptions and error handling outlines the general approach for dealing with error conditions and again is similar to other languages with exceptions.

If you've used other languages like Pascal, C, Java or C# you'll be able to skim read these sections (but take note of the additional features available) and move on to the more interesting constructs such as Records, Classes, Functions and Traits. As an experienced developer much of this will be very familiar - maybe not in syntax but in concept. The same or similar concepts exist in most OO languages. Though you might find first class functions new and refreshing and these have turned out to be essential to unlock other capabilities like pipeline processing.

If you get that far and the syntax layout and approach seems to fit the way you'd like to work, then EK9 has much more to offer in the same vein. However if the structure and the approach of this language does not feel right for you, take a look at other languages instead.

Experienced Developers

Both functions and classes are treated on equal footing, they were both built into the language from the outset and complement each other.

There are a number of new quite powerful ideas that as an experience developer you may find very useful. These may fill a gap in how you've solved problems in the past. It is in this area EK9 adds more value with extension by composition that pulls an architectural design approach into the language itself. The effect of this is quite surprising in terms of how you develop reusable elements that can be composed together rather than just relying on inheritance. This allows for much more re-use.

As a modern language you would expect extension through inheritance and EK9 does include this, but with EK9 you can also extend functions in a similar manner, indeed you can create an abstract function. With this approach it is possible to define something similar (but not the same) as a lambda with dynamic classes and dynamic functions. You can read the section on readable pipelines and pipelines if lambdas were used to understand why pure lambdas have not been included in EK9. This is by design, but it is likely to be a capability missed by some developers.

Functions can be passed around as variables; delegates as Microsoft would call them. This single idea has enormous implications when combined with classes; as it enables variation in functionality on a per object instances when a method can delegate functionality to a function. This means that to vary functionality in a class it is not always necessary to extend the class, just providing a different delegate function for specific methods means its behaviour can change. This, in combination with traits enables much more flexibity through extension by composition.

It is hoped that even the few concepts covered so far will provide experienced developers with additional mechanisms they can use to solve problems in a more elegant, succinct and reusable manner.

Whetting your appetite for more

While the above capabilities are similar to other languages, the areas below tend to start to diverge into something EK9 specific.

EK9 does include Generic/Template based development see this article for details if you are unsure what these terms mean. There are some limitations on how generics/templated types can be developed, defined and used, again these are pragmatic restrictions designed to limit complexity. Some developers may find this too limiting. But for most developers this should provide a set of capable constructs to write code that can be used with a range of types in a parametric polymorphic manner.

EK9 comes with a number of generic types out-of-the-box. These are not an after thought, they have been designed in from the outset - in fact the language needs them, they are:

Folding design patterns into the language

There will be some developers and programming language theorists that will argue that programming languages should be minimal. Creating a larger set of key words or concepts will expand the amount a developer needs to learn to be productive. Drawing design patterns and frameworks into the language should not be done; as patterns tend to come in and out of favour (or trend). There is much to be said for this point of view and there are many (most in fact) languages that take this approach. You can use Lisp or Clojure if you prefer to stick with a language with a minimal number of keywords and constructs.

But there are some powerful reasons to go against these recommendations in a controlled and careful manner. See the sections below on the details of which patterns have been included. Please remember it is not compulsary to use these features and capabilities! If you are developing web services for example you are going to have to deal with HTTP verbs so including those in the language itself means that you don't need to learn a third party API. In either case you had to know the verbs. Clearly if you are not developing web services you just don't need to learn the verbs at all.

Dispatcher

These next set of ideas pull design patterns into the language itself. Some of these capabilities do exist in terms of third party API's or techniques, Swift being an exception in terms of the following dispatch idea - as it does include it.

The introduction of a dispatcher method on a class is quite a good way to handle polymorphic types. In short you write a single method that is marked as a dispatcher that accepts some base type or trait and then you write multiple methods with the same name but that accept classes that extend the base type. The EK9 code then works out which method is the best match when you actually call the dispatcher method with some parameter.

The real advantage of this approach is that it removes the need for any sort of casting or instance of checking. In fact the sytnax to cast or determine the type of an Object does not even exist in EK9 and neither does the word null. EK9 provides other mechanisms that mean these concepts/ideas do not need to be used.

As you can now see without the dispatcher, EK9 would have been forced to introduce casting and instance of. Without operator overloading and the introduction of ? (is set) it would have to use null.

Dependency Injection (DI) and Inversion of Control (IOC)

If you've used Spring in the past or use it now, but feel its gone a little too far from its original idea of IOC and Dependency Injection but would like to use it in a more limited and constrained manner; then EK9 has a pragmatic approach with components/applications and dependency injection which is built in.

This is the main reason EK9 does not have any concept of static or global variables, by using IOC/DI there is no need for those two ideas to be introduced. Clearly it is still possible to access a singleton via the DI but it is more loosely bound. This mechanism also complements the constants construct in enabling more complex Object instances to be used throughout the software being developed.

In conjunction with the two concepts above; EK9 also includes Aspect Oriented Programming (AOP) and this has been designed to dovetail in with the dependency injection rather than being a separate tool kit. In effect it enables the developer to intercept methods calls made on a component and complete some processing both before and after a method call.

None of the above concepts are new - they are all available in different languages/API's and toolkits. What EK9 does, is bring them together with a simple syntax and actually compile the code. By incorporating IOC/DI; EK9 has simplified the nature of variable scope; removing global variables that are very fixed in naming. It is still possible to have a global variable if you have to, but only via IOC.

Importantly the IOC and Aspect Oriented language features have been deliberatly watered down and have had their scope of applicability constrained. This is designed to prevent overuse. Again; if this capability is too minimal then sticking with languages like Java or Kotlin and using Spring might suit you as a developer more than using EK9.

A drunkard's walk?

You may be thinking; "all of the above design patterns, no null, strange syntax and very specific constructs where just a class would do", this is a drunkard's walk? Picking up this and that on the way in an incoherent and haphazard manner. Does it have every 'buzz word', technique and paradigm possible in it?

Most large scale software uses all of the above techniques/ideas in some form or other, but it is always implemented in an inconsistent manner or requires so many additional dependencies that take real work to fuse together. Remember if you don't need a capability (like a tool in a tool box) don't use it.

New/revisited concepts

The ideas that follow are a reworking of existing concepts not always found in programming languages directly but found in Unix/Linux shell environments or third party components and API's.

Pipeline processing

Many OO API's have introduced the idea of chaining methods, filtering, mapping and reducing with just normal class methods in what's called method chaining or a fluent interface both of which are similar in concept to the creation by Douglas McIlroy called a pipeline for processing text files on the Unix operating system (really a very clever idea).

So the idea of streaming Objects and streaming/piping file contents will be familiar to many developers.

EK9 combines both of these concepts and functional programming together with the Monad type concept from the Haskell language. The purpose is to enable a stream of Objects (and Function delegates) to be sent through a notional processing pipeline; where upon a number of commands (listed below) can be applied in conjunction with (in some cases) a function to control either the flow or to transform the contents or type of Object that continues down the pipeline.

The streaming mechanism uses the pipe '|' symbol for joining parts of the pipeline just like Unix and it includes:

But also adds:

The terminal part of a pipeline can be used to collect the results.

If you've ever used Unix/Linux shell commands like 'ps -fe | grep mysql' then you will feel right at home with EK9 pipelines.

All of the above are described and explained in the section on Streams/Pipelines you may find this set of capabilities one of the main reasons to consider EK9. It removes the need for complex nested for loops and iteration over collections (though this can still be done). It presents the processing in a readable format and it enables re-use of functions that have a simple single testable (and pure) purpose. But best of all it removes the need for endless null checking (idea from Haskell). EK9 does not really use null, it employs the idea of is set; see the section on operators for more details on this.

This pipeline processing concept has been introduced to EK9 for a couple of reasons. First, the Unix concept of the pipeline has remained in place since the 1970's; so as a concept it has proven its worth and is very valuable. The syntax is familiar, easy to read and understand; but is also terse and to the point.

The functional programming design approach is now once again more accepted in software development (for some it never went away). From the Object Oriented world; the miriad of different methods put on different API's in other languages to accomplish the same or very similar functionality in terms of Object streaming leads to confusion/repetition/inconsistency and is different from API to API.

This is an important point (with which many may disagree) an Object Oriented technology and technique has been misused to create a functional programming flow. It's like using a screw driver as a chisel - you can - but why not just use a chisel if that's what's needed?

Pulling these ideas in to the language together with the (Haskell concepts of Maybe/Monad) is set nature of Objects and Generics (containers of zero or more Objects) enables a standard syntax to be established. It also promotes re-use of functions as solid testable building blocks.

There is no attempt here to state that a functional approach is better or worse than an Object Oriented approach; each has strengths and weaknesses, EK9 attempts to enable the developer to use the strength of each approach in coordination/harmony.

Threads

If you are a more experienced developer you may be thinking does EK9 use Threads or Cooperative multi-tasking (like Javascript). EK9 assumes Threaded implementations and as you can see in the above pipeline outline introduces async. This enables multiple functions to be called at the same time (asynchronously).

This is not the only place that Threads are alluded to. In the Network Types section the TCP Server and to some degree the UDP server are designed to accept multiple concurrent clients (as you would expect). The capability of signal handling also implies multi-threading. To support this implied multi-threading EK9 does have a Mutex Lock built in, should you need to use it (clearly where possible avoid simultaneous access to data structures - as it leads to complexity, defects and performance issues).

Without the idea of Objects and Generics there could be no Lists and without Lists, Functions and is set there could be no Pipelines. Each construct is equally important, but each is used an an appropriate way.

Preview

Here is a sneak preview so you can really see how EK9 has forged Unix pipeline ideas with Functional Programming, Monads and an Object Oriented approach. In this example cat is the command to start streaming a collection of something, people is a List of Person (Person is a record), lastNameAndDob, lastNameComparison etc are all functions and stdout is an instance of the Stdout class that can be used to print Strings to the terminal output.

Readable pipeline
  • cat people | uniq by lastNameAndDob | sort with lastNameComparison | group by personProfession | sort by professionComparison | map with personToString > stdout

This summary should give you some idea of what EK9 has to offer. The syntax above is very simple to understand and the pipeline flow is obvious in terms of readability and concept.

There is an alternate layout for longer pipelines which makes the pipeline even easier to read.

  • cat people
  •   | uniq by lastNameAndDob
  •   | sort with lastNameComparison
  •   | group by personProfession
  •   | sort by professionComparison
  •   | map with personToString
  •   > stdout

Readability/Testability is the main reason EK9 does not promote in-line lambdas. The implementation detail of lastNameAndDob is shown below (the '$' is just the operator to return a String version of the variable - the person's last name and date of birth).

The column layout (and short line lengths) always makes code easier to read. Albeit at the cost of longer methods and functions.

'lastNameAndDob' function implementation
  • $ p.last + " " + $ p.dob

If in-line lambdas had been allowed in the pipeline itself, it would have been necessary to state what the name of the variable being accepted was (in the case above 'p') and some new operator '=>'. So an excerpt of the above example would now read as:

Obfuscated pipeline (not supported)
  • cat people | uniq by (p) => $ p.last + " " + $ p.dob | sort with lastNameComparison | group by personProfession | sort by professionComparison | map with personToString > stdout

You will probably find you are drawn into the lastNameAndDob implementation part. That distracts you from the overall flow of the pipeline! You then lose your place in what the objective of the pipeline was; due to the distraction.

Now could you imagine seeing the full pipeline processing with all those implementation details exposed as in-line lambdas it would be much harder to comprehend, test, maintain and alter, never mind having to include more complex expressions over multiple lines.

If each operation required has to be a function - it is now simple to write a unit test to check that function works. If you in-line the functionality in a lambda you cannot test that part in isolation, moreover no inline lambdas is re-usable. In general this leads to copy-paste and lack of maintainablity and is not DRY.

Hopefully you can see how the concepts of generic collections, record/class objects and functions can all be combined together where each construct is used to its strength and control/processing is passed between these concepts to create a solution that is readable, re-usable, simple, maintainable and strongly typed.

The approach taken in EK9 leads to clarity of the pipeline and function re-use. The additional cost in terms of writing the functions rather than in-line lambdas is acknowledged and accepted.

Web Services

As you would expect EK9 includes standard libraries to make calls to web resources using GET, POST, PUT, PATCH, HEAD and DELETE with just a simple call to a WebResource.

But it also has support built into the language to define web services/resources, i.e HTTP end points. In general these sorts of higher level capabilities are provided by third party API's with bespoke Annotations; but HTTP and specifically REST/RESTful web services are here to stay. For more information on REST you can read the original dissertation by Roy Thomas Fielding (Dr).

Those technologies that were once just an aside to mainstream programming are now a fundamental building block. EK9 treats them as such and hence pulls them into the language itself. Below is a quick preview of what that syntax looks like. This just shows a simple GET, but the section on Web Services shows the use of all the verbs. Make sure to read the section on operators first as POST uses + (+= could also be used for POST) to support 'adding' new resources via HTTP/REST/RESTful web services; this fits in with the other Object Oriented approaches in EK9 to collections.

Clearly it is possible to go beyond CRUD (RPC) and implement level 3 REST with HATEOAS, this is driven by actual content. EK9 just provides the basics with respect to HTTP and VERBS.

#!ek9
defines module introduction
  defines service
  
    //define the service called 'Addresses' and map it to a URI of '/addresses'
    Addresses :/addresses

      //A GET  
      listByPostCode() :/{postCode}/
        ->
          postCode as String //Now assume PATH PARAM
          acceptLanguage as String :=: HEADER "Accept-Language"
        <-
          response as HttpResponse: () of HTTPResponse
        //override HTTPResponse body, status and etag to complete procesing.
//EOF

The key concept in REST is the resource with EK9 you can use any type of content you wish and serialise it from any of the EK9 constructs. Typically this would normally involve records and/or classes as these are constructs that hold state. But it would be quite possible to just employ a String that conformed to a particular layout if that was appropriate (i.e JSON).

EK9 philosophy (or approach)

The general approach of the development of the EK9 language has been to blend many existing language constructs, concepts, design patterns or even syntax in some cases in a gentle and cohesive manner. This means that in many cases the EK9 approach to the same sort of language construct is not always as powerful, terse or flexible as in other languages.

However when several of these approaches are linked in combination, the overall solution is more explicit, simpler to develop, more powerful overall and re-useable; but importantly it remains readable. So in general the focus has been on trying to make the resulting code reasonably terse, understandable and reusable - over just being quick to implement in the first instance. Long lived code is always read many more times than it is written.

The EK9 language errs on the side of strong typing, readability, re-use and composition over terseness. List comprehensions are not included in EK9 for this very reason. The same functionality can be delivered with pipelines.

Lambdas

Another concrete example of this, is the approach to lambdas. EK9 enforces the separation of the mechanism of implementation of the functionality from where and how it is used (i.e abstraction). So including the details of how to implement the lambda in the same context of where it is used flies in the face of the EK9 approach. You can see the contrast here in the two examples function pipeline and in-line lambda pipeline.

There is a slight compromise here through the use of abstract functions, see DynamicMathExample2 as this does in-line a dynamic function. It has to be a single line function however.

In a complex pipeline it is best to be able to focus on the pipeline construct and not be distracted by low level implementation details in the lambda. Something like this simple operation (the proposed lambda) is very likely to be used elsewhere so why not just define a function so it can be reused. If the operation is more complex then even more reason write a function to describe what it does and make it testable.

It is accepted that in-line lambdas will be missed by some developers. EK9 provides dynamic functions; which are similar to lambdas in some ways. You may ask why make the compromise for single line dynamic functions when they extend abstract functions, well just look at examples DynamicMathExample1 and DynamicMathExample2.

Designing a programming language is all about compromises. Through the use of abstract functions and their inferred input and output parameters, the main issue of having to declare incoming and returning parameters is mitigated.

But only by writting some code in EK9 and doing the worked examples did it become obvious that this compromised should be accommodated.

Break and Return

EK9 does not use break or return to facilitate flow control. Functions and Methods can return values, but when the return takes place is always at the end of the scope block. Loops cannot be curtailed early and broken out of. You may consider this an unnecessary constraint and of course like most things in software design it is subject to much heated debate.

The addition of break and return would add to the flow control statements of if/elseif/else and switch and if we also consider try/catch/finally and exceptions as a flow control statement; there are already many different ways to control processing flow.

The reasoning (which you may not agree with) for omitting those key words is as follow: Developers wish to 'return' early is because the coding approach is a 'procedural' one, likewise 'breaking' out of a loop early is the same 'procedural' approach. The biggest disadvantage of this approach is the level of complexity in understanding of how the flow path through the whole code block will be followed (by a human, not a compiler).

By including the additional flow control mechanisms; code becomes harder to reason and refactor/modify (opinion). In some cases processing logic that is written towards the end of a function or method is no longer executed (because of the early break or return). This means that the developer has a much harder job to understand the flow logic (rather than just with if/elseif/else and switch for example).

Break and Return Alternative

Rather than take a procedural approach with many for loops, if/else logic; EK9 has the concept of pipeline processing. It is this; that is the alternative to using early returns and breaking out of loops. By using combinations of filter, head, tail and skip - there is a standard syntax that means you can accomplish the same outcome but in a more maintainable, consistent and obvious syntax. Though it is accepted that some developers may find this initially constraining/frustrating. This approach does lead to a larger number of small discrete functions. This promotes re-use and unit testing and by its very nature forces documentation in the form of a function name (ideally a meaningful name).

User Interface

Many developers will want to use alternative front end frameworks and not adopt EK9 for front end development. This is the main reason the EK9 language has Web Services built in. Existing front ends can remain as is and just connect to an EK9 developed micro service using REST or in a RESTful manner.

Front end development is really still in a state of flux, the native UI, mobile devices and WEB are all pulling front end development in different directions (still in 2021). UI development can and probably will remain outside of the language itself. But through the use of HTTP and Services the language can support the development UI tool kits.

SQL

You may be wondering why (or relieved) EK9 does not attempt to abstract a data persistence layer. The main reason for this is that data persistence in the form of SQL tables present an inherent mismatch in terms of Objects (EK9 is also not 'OO' focused). Moreover with the rise of No SQL databases there are alternative persistence mechanisms with very different semantics and concepts. Do not assume that the record construct in EK9 always equates to a record in a database table. It could if that is what the developer intends but it does not have to be. The record construct can represent any aggregate of data, this could be pulled together from a wide range of sources.

Experience has shown that introducing something like an abstraction layer for SQL does have immediate benefits; but for larger scale software with a long life time that incorporates OLTP and BATCH type functions, use of native SQL (or the NoSQL equivalent) is almost always inevitable. This then leads to writing and rewriting the same or similar functionality in both the abstracted form and SQL/NoSQL (not DRY).

Moreover it means you cannot use pure SQL/NoSQL resources both written and personnel to improve, optimise and debug database queries. The abstraction layer for database interactions almost always leads to needing to see what SQL/NoSQL the abstract layer generated for debugging or performance issues.

Therefore EK9 does not have a hibernate layer. Just as with HTML/CSS; EK9 accepts the fact that most developers will (and will want to) use SQL/NoSQL in the dialect of their choice for the database of their choice.

Libraries

While it is hoped that a very wide range of libraries will be developed for EK9 it is also understood that many systems are being developed as microservices. With each service being written in a language appropriate for the functionality and development team. The enforced creation of very large monolithic applications just to be able to re-use existing libraries is now behind us. EK9 does not define discovery services or even mechanisms to announce the presense of a microservice. This activity is considered to be part of the configuration, management and deployment of software.

As EK9 is designed to fit in with other languages and services it is expected that 'DevOps' staff will have their own preferred way to deploy and manage mircoservices. For example the developer of a microservice could create the service so that is can be command line or environment variable driven, so the port it services can be configured. This gives the 'DevOps' staff a familiar and obvious mechanism to configure and start the microservice. The use of configuration files for a service should only be considered then; if there are many options for configuration.

Perhaps a configuration data microservice should be considered, this could provide reference configurations for some microservices and again breaks a single microservice down into two (or more). If the same configuration is applied to all instances of a customer microservice for example, then all instances of the microservice have access to the configuration. This also means that the amount of configuration required to start each instance is minimised.

EK9 is not prescriptive on how application/microservice configuration is managed and importantly it does not introduce yet more diverse concepts and ideas for software deployment and management.

Conclusion

The EK9 language has been developed over a number of years by looking at what techniques/technologies work and what don't. It is hoped that the syntax, mechanisms and layered approach in EK9 will provide developers with a range of different tools. This should enable software to be developed in a flexible, pragmatic, maintainable and clear fashion.

Clearly there will be some developers with similar experiences that may have drawn the same conclusions. There will be some that strongly disagree with the direction this language has taken. There is ample choice of language for all, it is best to select one that suits your experience, nature and the project requirements.

The very nature of many applications is now one of micro services in some caes running in containers being fronted by caching, load-balancers and fa├žades.

Finally

It is accepted that the EK9 language is big and is someways quite a departure from existing languages. But as stated before, to gain wide spread adoption it has to provide real and significant benefits in comparison to other languages. It can't be just "a bit easier/better here and there".

Hopefully you will notice that the overall layout has a number of general aspects to it.

Initially when you look at some of the code examples - it might take you a while to get your 'eye in'. Hopefully your brain will suddenly get used to indentation and → and ← and the data processing flow will just visually 'click'.

You should find you can 'step back' from the code without reading the words and just 'see' the structure - like focusing on shape of woods rather than each tree. It sounds a bit hippy and new-age, but there has been real focus on creating a syntax and file layout that is pleasing on the eye and aims to make code readable for humans (indentation, columns, structure and direction). This is why there has been a focus on operators as these or concepts that are taught in schools from an early age. i.e the mathematical operators of +/- for example.

Next Steps

Go to the section on structure for much more detail on the constructs outlined above.

Other Languages

There are many other languages that might be more appropriate for specific programming tasks. So before looking into EK9 its probably worth looking at what sort of target solution you are looking to develop.

Compiled to produce native applications

Compiled and strongly typed languages that use a runtime

The decisions on whether to use Java/Scala/Kotlin (all of which target a JVM) really comes down to preference and balance - there are really no big issues with selecting any one of those languages.

C# on the other hand really is a major decision - you move well into the Microsoft camp with C#, it is a very, very good language and addresses many of the issues that have emerged with use in Java.

Other Compiled and strongly typed languages of note

Interpreted/Scripted Languages

Other Languages worth considering

Next Steps

If you'd still like to understand more about EK9 look at section structure next.