Collection Types

This section just focuses on the types that are used as collections; some of which are generic/template types. Collections hold zero or more Objects of the same or compatible type.

The Types

Supporting Types

List

Some of the examples in section built-in-types have used the List type. It is quite a simple type to use as it really just holds zero or more Objects of a specific type. It supports the following operations/operators:

You can make a List of any type (and that includes functions), so for example if you had an abstract function for collecting data from a range of different sources, you could create a list of those different functions and then use them via their abstract signature.

The example show how Lists can be created - in this case it is just a simple List of Integer. Note there is no need for '<' or '>', just declare a List of X, where X is the type. Where the type has two parameteric parameters (like Dict), use '(' and ')' i.e. Dict of (S, T) to group the parameters.

#!ek9
defines module introduction

  defines function
  
    toString
      -> item as Integer
      <- rtn as String: $item
      
    isSet
      -> item as Integer
      <- rtn as Boolean: item?
        
  defines program
    
    ListCreationExample()
      stdout <- Stdout()
    
      //A couple of ways to create and empty List of Integer        
      aList as List of Integer: List()
      bList <- List() of Integer
      
      //But if you have a value you want to use from the outset then use this mechanism
      //The compiler can infer the type is a List of Integer from this.
      cList <- List(1)
      
      //If you have a full set of integers you know from the outset then you can use
      //[...] as a short hand for Lists (but they are not arrays and cannot be indexed with []).
      l1 as List of Integer: [ 57, 55, 26, 24, 21 ]      
      
      //This inferred declaration is preferred and more minimal      
      l2 <- [ 1, 2, 3, 4, 5, 6, 7, Integer() ]
      
      //List has a $ operator and so can just print the contents out.
      stdout.println("List l1 is " + $l1)      
      stdout.println("List l2 is " + $l2)
      
      //Clearly if you had many items, the line length would get very long,
      //so you can declare lists like this. Very useful if the items in the list have
      //complex constructors.
      l3 <- [
        1,
        2,
        3,
        4,
        5,
        6,
        7,
        Integer()
        ]
      
      //Again you can just print out like this
      stdout.println("List l3 is " + $l3)
      
      //But the alternative is to use streaming
      cat l3 | filter by isSet | map with toString > stdout   
//EOF

As you can see there are a number of ways to create Lists and use them. The outputs from the above would be:

Importantly as Integer() was used to populate an entry it is not set as you can see in l2 and l3 above (trailing ','). If it is important to only process valid values then you can use the stream pipeline processing as shown above. This will drop the final unset value in list l3.

The example is quite simple and really just shows the range of different ways that Lists can be declared. The following example is aimed at demonstrating how EK9 functions as delegates that can be treated as Object in a polymorphic manner but in the context of Lists and stream pipelines. It also shows the record construct.

#!ek9
defines module introduction
  defines record
  
    //Designed to hold members of the Addams family only
    Addams
      name String: String()
      dob Date: Date()
      
      //Sample of a of developer constructor, other built in ones get auto generated.
      Addams()
        -> dateOfBirth as Date
        if dateOfBirth?
          dob :=: dateOfBirth
                 
      operator $
        <- rtn as String: name + " " + $dob
        
      operator ?
        <- rtn as Boolean: name? and dob?
        
      operator #^
        <- rtn as String: $this
        
  defines function
  
    //Just convert Addams family member to a String representation
    addamsToString()
      -> person as Addams
      <- rtn as String: `DOB: ${person.dob} Name: ${person.name}`
          
    //Polymorphic abstract function for getting Addams family members   
    getAddams() as abstract
      <- rtn as Addams
    
    //Using the full constructor that is auto generated.  
    getMorticia() is getAddams
      <- rtn as Addams: Addams("Morticia", 1965-01-03)      
    
    //Use developer defined constructor then set the name
    getGomez() is getAddams
      <- rtn as Addams: Addams(1963-06-08)
      rtn.name: "Gomez"
      
  //Concept to hold functions that can get Addams family members
  //Then when the time is right actually call the function to get the member.
  defines program
    
    ListFunctionIteratorExample()
      stdout <- Stdout()
      
      //Note this is a List of getAddams (function)
      //EK9 is smart enough to see they both have a common super (function signature)
      functionList <- [ getGomez, getMorticia ]
      iter <- functionList.iterator()
      
      //A bit long winded - see next stream pipeline example 
      while iter?
        fn <- iter.next()
        //Now make the call and convert the returned value to a String for output
        stdout.println($fn())
        //Because Addams has the promote operator #^ to a String we could just do this
        stdout.println(fn())
     
    ListFunctionStreamExample()
      stdout <- Stdout()
      
      //Again a list of functions
      functionList <- [ getGomez, getMorticia ]
      
      //But here we pipe through to 'call' the function and get the return value
      //We use the fact that promotion can be used to convert to a default String
      cat functionList | call > stdout
      
      //Now if the calls to get members of the Addams family via the function
      //Had to make remote or slow calls, you might be better off calling them concurrently
      //In an asynchronous manner - note the pipeline will wait until all the calls have completed
      //But also in this case we want to control to conversion to a String by using a function.
      cat functionList | async | map with addamsToString > stdout

      //It is also possible to just use this, rather than creating a variable to reference the list
      cat [ getGomez, getMorticia ] | async | map with addamsToString > stdout

//EOF

The example above should give you a better idea of how Lists can be used, but also how using functions in a polymorphic and Object Oriented manner via delegates means that when used in conjunction with stream pipelines you can defer processing and also call functions asynchronously.

The above example is quite important in terms of using EK9 and mixing and matching a functional approach with an object oriented one. Linked with type inference, generics and stream pipelines the processing becomes more declarative in how the processing is accomplished. This can be seen with the two examples above. ListFunctionIteratorExample is more procedural than ListFunctionStreamExample. However the development thought process is a little more sophisticated; as is always the case with a declarative approach.

Here are those two example bodies without the comments.

  • iter functionList.iterator()
  • while iter ?
  •   fn iter.next()
  •   stdout.println(fn())
  • //Versus
  • cat functionList | call | map with addamsToString > stdout
  • //Or (if we provide and use the #^ 'promote' operator)
  • cat functionList | call > stdout

Clearly the function addamsToString still has to be written (unless we provide the promote operator), but in keeping with the philosophy outlined in the approach section this overhead is accepted. In this particular case being able to change how the String representation should be output is important in creating maintainable/easy to improve code.

As you can see in the example above, there is quite a lot of flexibility on where and how to do the processing, use of operators, functions to alter and transform aggregate object data as it is passed through the processing pipeline.

Optional

While it is possible for each type to define isSet via the '?' operator sometimes it is also necessary to wrap an object inside a wrapper that may or may not contain that type of object. This is what Optional is aimed at doing.

In the example below Optional is used with the Integer type but it can be used with any type or even a function delegate or record. The example is designed to show how you can use the Optional in conjunction with other types and how List and pipeline streams can be used with flatten to extract the values out of the Optional but in a safe manner. Note that while Optional does have a get() method it is not shown in the example, this is to try and encourage the use of the other mechanisms that are much safer than get(). Because if the Optional does not hold a value then an exception will occur. Whereas in the examples below the flatten and iterator() mechanisms are all safe and provide the value from the Optional only if it is held.

The concept of Optional, flatten and pipelining is a mix of ideas (Monad) from the Haskell programming language, Unix 'pipes' and other main stream programming languages. The idea of having containers with zero or one value is actually of value. EK9 has attempted to blend these ideas together in a pragmatic manner.

#!ek9
defines module introduction

  defines program

    ShowOptionalType()
      stdout <- Stdout()

      //Example with a fully qualified declaration
      maybeInteger as Optional of Integer: Optional()

      //Nothing will be output
      cat maybeInteger > stdout

      //A type inferred declaration
      item1 <- Optional(5)
      cat item1 > stdout

      //two different ways of checking if optional has a value
      assert item1 is not empty
      assert item1?

      //read as List of (Optional of Integer)
      //fully qualified declaration
      emptyOptionals as List of Optional of Integer: List()

      //type inference mechanism - much more terse
      aListOfOptionals <- List(maybeInteger)
      aListOfOptionals += item1
      aListOfOptionals += Optional(14)

      //But could have been defined like this
      someOptionals <- [maybeInteger, item1, Optional(14)]

      //Use of flatten to get the items out of the optional
      cat aListOfOptionals | flatten > stdout

      sum <- cat aListOfOptionals | flatten | collect as Integer
      assert sum == 19

      //Also possible to use an iterator if you wish
      iter <- item1.iterator()
      while iter is not empty //could have been just 'while iter?'
        stdout.println("Value [" + $iter.next() + "]")

      //remove the item that was within the optional
      item1.clear()
      assert item1 is empty
      assert not item1?
//EOF

The example above (while focusing on Optional) also alludes to the Haskel (monad-ish) flatten and maybe nature of Lists and Optionals by providing a null safe way to pull data out of wrappers like Lists and Optionals. The alternative would be a range of loops and null (isSet) checks.

  • sum ← cat aListOfOptionals | flatten | collect as Integer

The above snip from the example highlights the power of using the Generic collection types with Stream pipelines and the built in standard syntax of cat, filter, map, collect etc. In effect the above works as a reduce operation to sum the valid Integers that flow through the pipeline. But takes into account the list could be empty or some of the optionals might not actually hold any values.

Iterator

The previous examples of both List and Optional have shown the Iterator. It is really only useful in being able to provide a mechanism to step through zero or more items held in some type of collection. The main methods are '?' empty() and next(). You can see in the above examples iter? can be used or iter is not empty second is more readable but obviously less terse.

Bits

The Bits type has been covered in built-in-types, it can be viewed as an ordered list of Booleans.

String

The String type has been covered in built-in-types, it can be viewed as an ordered list of Characters.

Strings

The Strings type was introduced in built-in-types, it can be viewed as an ordered list of Strings. This type has been created because Lists of String are quite pervasive and if necessary can be converted to an actual List with ease.

A short example of such a conversion is shown below.

#!ek9
defines module introduction

  defines type
    List of String

  defines program

    ShowStringsType()

      s1 <- "First"
      s2 <- "Second"
      space <- " "

      asListOfStrings <- [s1, space, s2]

      asStrings <- cat asListOfStrings | collect as Strings

      backToListOfStrings <- cat asStrings | collect as List of String
//EOF

PriorityQueue

This is a form of List but can be ordered and can also be finite in size. Very useful for keeping a list of items in ordered form. If the size is not set then all items added to the queue will remain in the queue. If no ordering is specified then the order is random when accessed (and items will not be in the order they were added).

Once the size is set then number of items in the list will remain below or equal to that size. Once again if no ordering is specified then random elements are removed.

The real value of this collection is when both size and ordering are specified. Then; as is shown in the example below, a finite number of elements are retained but in the priority order specified.

#!ek9
defines module introduction

  defines type
    PriorityQueue of String

  defines function

    <?-
      Used with filters, anything over 5 characters in length is suitable.
    -?>
    suitableLength()
      -> value as String
      <- rtn as Boolean: length of value > 5

  defines class

    <?-
      Provide in reverse comparison order.
    -?>
    StringCompare is Comparator of String
      override call()
        ->
          o1 as String
          o2 as String
        <-
          rtn as Integer: ~(o1 <=> o2)

  defines program

    ShowPriorityQueue()

      qOne <- PriorityQueue("Bill").useComparator(StringCompare())

      qOne += "And"
      qOne += "Ted"
      qOne += "Excellent"
      qOne += "Adventure"
      qOne += "Outrageous"
      qOne += "Bogus"

      allEntries <- cat qOne | collect as List of String
      //Check they are in reverse order by String comparison not order added.
      assert $allEntries == "[Ted, Outrageous, Excellent, Bogus, Bill, And, Adventure]"

      //Now make the list finite with just 3 entries
      qOne.setSize(3)

      //Access to a list can be done just like this
      limitedEntries <- qOne.list()
      assert $limitedEntries == "[Bill, And, Adventure]"

      //Not sure Missy will make the cut!
      qOne += "Missy"

      //Get the list again but in reverse order.
      asList <- ~qOne.list()
      assert $asList == "[Adventure, And, Bill]" //No 'Missy' did not make it.

      //Use PriorityQueue to collect up and keep ordered as part of pipeline
      bestTwo as PriorityQueue of String: PriorityQueue().setSize(2).useComparator(StringCompare())

      //Note this uses allEntries again, but filter before queueing
      cat allEntries | filter by suitableLength > bestTwo

      //Check the string representation or the reverse ordered list.
      //'Outrageous' qualified in the filter, but was not in the top two when ordered in the queue
      assert $~bestTwo.list() == "[Adventure, Excellent]"

//EOF

Clearly the real power of the PriorityQueue comes when you need an ordered list that is finite in size, especially when processing a large stream of data items. The example above gives a brief view of how the PriorityQueue can be used as the end of a Stream pipeline to gather up data in an ordered a finite collection.

You could imagine having a number of PriorityQueue objects and need to merge them in some scenarios. Given a List of PriorityQueue of String a simple pipelines with flatten and a final '>' into a suitably sized and order PriorityQueue would be simple and obvious to implement.

As an aside, EK9 takes the approach of building small distinct functions and classes so that they can be combined in components, compositions and pipelines to accomplish processing; as opposed to providing many capabilities just within classes.

Dict (Dictionary/Map)

There is quite a long example of the use of Dict (Dictionary/Map) and DictEntry below. It also aims to demonstrate how Dictionaries can be used with functions and processing pipelines. It also shows how dynamic functions can be employed.

Importantly the Dict is parameterised with two parameters, with Generic/Template functions/classes in EK9 it is necessary to use '(' ')' where more than one parameter is used. This is to ensure that the generic types being specified are not ambiguous.

DictEntry (Dictionary Entry)

Example shown below.

#!ek9
defines module introduction

  defines type
    List of getAddams
    Dict of (Integer, String)
    Dict of (String, Date)
    Dict of (Integer, Addams)
    Dict of (Integer, getAddams)

  defines record
    Addams
      name String: String()
      dob Date: Date()

      Addams()
        -> dateOfBirth as Date
        if dateOfBirth?
          dob :=: dateOfBirth

      operator $
        <- rtn as String: `${name} ${dob}`

      operator ?
        <- rtn as Boolean: name? and dob?

      operator #^
        <- rtn as String: $this

  defines function
    idToString() as abstract
      -> id as Integer
      <- rtn as Optional of String: Optional()

    nameToDate() as abstract
      -> name as String
      <- rtn as Optional of Date: Optional()

    idToAddams() as abstract
      -> id as Integer
      <- rtn as Optional of Addams: Optional()

    idToGetAddams() as abstract
      -> id as Integer
      <- rtn as Optional of getAddams: Optional()

    addamsToString()
      -> person as Addams
      <- rtn as String: `DOB: ${person.dob} Name: ${person.name}`

    getAddams() as abstract
      <- rtn as Addams

    getMorticia() is getAddams
      <- rtn as Addams: Addams("Morticia", 1965-01-03)

    getGomez() is getAddams
      <- rtn as Addams: Addams( "Gomez", 1963-06-08)

    getPugsley() is getAddams
      <- rtn as Addams: Addams("Pugsley", 1984-10-21)

    getFester() is getAddams
      <- rtn as Addams: Addams(1960-11-21)
      rtn.name: "Fester"

    getWednesday() is getAddams
      <- rtn as Addams: Addams()
      rtn.name: "Wednesday"
      rtn.dob: 1998-01-09

  defines program
    ListDictionaryExample()
      dictionaryExample <- SimpleDictionaryUse()
      dictionaryExample.showDictionary()

  defines class
    SimpleDictionaryUse
      private getName()
        <- name String: "Gomez"

      showDictionary()
        idNameDictionary <- { 1: getName(), 2: getMorticia().name, 3: "Pugsley", 4: "Fester", 5: "Wednesday" }

        addamsParents <- {
          getName(): 1963-06-08,
          getMorticia().name: getMorticia().dob
          }

        addamsKids <- {
          "Pugsley": 1984-10-21,
          "Wednesday": 1998-01-09
          }

        nameDateDictionary <- addamsParents + addamsKids + DictEntry("Fester", 1960-11-21)

        idRecordDictionary <- {
          1: getGomez(),
          2: getMorticia(),
          3: getPugsley(),
          4: getFester(),
          5: getWednesday()
          }

        idFunctionDictionary <- {
          1: getGomez,
          2: getMorticia,
          3: getPugsley,
          4: getFester,
          5: getWednesday
          }

        results as List of String: List()
        keyIter <- idNameDictionary.keys()
        while keyIter is not empty
          key <- keyIter.next()
          name <- idNameDictionary.get(key)
          if name?
            results += `Key ${key} with name ${name.get()}`

        //Some keys that won't ever be found
        invalidValues <- [9, 10, 100]

        keys1 <- idNameDictionary.keys()
        keys2 <- invalidValues.iterator()

        nameMapping <- (idNameDictionary) is idToString
          rtn: idNameDictionary.get(id)

        validValues <- cat keys1, keys2 | map with nameMapping | flatten | collect as List of String

        assert $validValues == "[Gomez, Morticia, Pugsley, Fester, Wednesday]"

        dateMapping <- (nameDateDictionary) extends nameToDate
          rtn: nameDateDictionary.get(name)

        //Now we've used the previous iterator up so need new ones
        keys1A <- idNameDictionary.keys()
        validDates <- cat keys1A
          | map nameMapping
          | flatten
          | map dateMapping
          | flatten
          |collect as List of Date

        assert $validDates == "[1963-06-08, 1965-01-03, 1984-10-21, 1960-11-21, 1998-01-09]"

        recordMapping <- (idRecordDictionary) extends idToAddams
          rtn: idRecordDictionary.get(id)

        //Now we've used the previous iterator up so need new ones
        keys1B <- idNameDictionary.keys()
        validAddams <- cat keys1B
          | map recordMapping
          | flatten
          | collect as List of Addams
          
        //"[Gomez 1963-06-08, Morticia 1965-01-03, Pugsley 1984-10-21, Fester 1960-11-21, Wednesday 1998-01-09]"

        //Example of an inline dynamic function        
        keys1C <- idNameDictionary.keys()
        addamsFamily <- cat keys1C
          | map with (idFunctionDictionary) extends idToGetAddams (rtn: idFunctionDictionary.get(id))
          | flatten
          | call
          | collect as List of Addams
          
        //"[Gomez 1963-06-08, Morticia 1965-01-03, Pugsley 1984-10-21, Fester 1960-11-21, Wednesday 1998-01-09]"

        values <- idNameDictionary.values()
        assert $values == "[Gomez, Morticia, Pugsley, Fester, Wednesday]"

        found <- idFunctionDictionary.get(1)
        assert found?
        m <- found.get()
        assert $m() == "Gomez 1963-06-08"
//EOF

Quite a bit of the above example has been used previously; the Addams record and the concept of abstract functions.

The snip below from the example shows the shorthand way of defining and populating the dictionary (Dict) entries. This shows a Dict with key of type Integer and value of type String.

  • idNameDictionary ← { 1 :getName(), 2 :getMorticia().name, 3 :"Pugsley", 4 :"Fester", 5 :"Wednesday" }

Sometimes for longer dictionaries the following layout is more suitable.

  • addamsKids ← {
  •   "Pugsley": 1984-10-21,
  •   "Wednesday": 1998-01-09
  •   }

Dictionaries (Dicts) can be built up in the following manner.

  • nameDateDictionary addamsParents + addamsKids + DictEntry("Fester", 1960-11-21)

As you can see from idRecordDictionary and idFunctionDictionary it is possible to have dictionaries that have values of records or even function delegates. It is also possible to iterate over the keys or the values of the dictionary. One of the more interesting parts of the example is how a dictionary is wrapped in a dynamic function, as shown below.

  • nameMapping ← (idNameDictionary) is idToString
  •   rtn : idNameDictionary.get(id)

While dynamic functions have not really been covered get, this above shows how a function can capture variables (a dictionary in this case), and how both the incoming and return variables are implicit and do not need to be declared. This shows the polymorphic nature of functions as this dynamic function extends the type (abstract function) idToString. All the function does is accept the incoming parameter id and look it up in the dictionary.

The use of the dynamic function is shown below, here two iterators are used as the source of Integer 'ids'. One source is the iterator of the keys from the dictionary, the second source is an iterator from a List of Integer where no entry in the dictionary exists. This example is designed to highlight the safe way to access and pipeline information. The nameMapping function will return and empty Optional and flatten will check this and ensure no invalid value gets passed on to be collected.

  • cat keys1, keys2 | map with nameMapping | flatten | collect as List of Integer

The snip above highlights the re-use and stand patterns that can emerge with pipeline processing. You can see the other pipelines in the example all follow the same pattern only the types and functions employed are different; in the case where function delegates are the type the call or async command can be used to actually execute the function to get the return value.

Summary

As you can see from the examples above; the collection types and stream pipelines when linked with functions create a powerful and consistent combination. The resulting stream pipeline syntax has been designed to be separate for methods on Objects and Collections.

Next Steps

That is the end of the collection types, the next section on Standard Types covers classes that really provide functionality as part of the standard API that comes with EK9. These types are very likely to grow in number and capability.