I am designing a J-like language I call JWithATwist. Some design questions are under discussion and I will describe them here. I welcome comments. One such design question is the functionality of scalar operations.
A noun in J is an n-dimensional array. Each element in this array can be a simple value, but it can also recursively contain other n-dimensional arrays. The number of dimensions of this array is called its rank. An array of the lengths in each dimension is called its shape.
When you write x + y in APL, J and similar languages it does not simply mean that two values are added. In JWithATwist x + y means that:
- If the left and right noun arguments have the same shape the elements in corresponding positions in the argument arrays are added. This operation is called map in many functional languages.
- If one of the arguments is scalar, a single item, this item is added to each of the items in the other argument.
- If the arguments have different shapes, then if the shapes have a common left part, the argument with the shorter shape is logically replicated to the same shape as the larger argument and the operation is performed between corresponding elements. When a row of two elements is added to a table of two rows the first element in the row of two elements is added to each element in the first row of the table of two rows, and the second element of the row of two elements is added to each element in the second row of the table of two rows.
The first two cases can be seen as special cases of the third case. A special case in which the left common part is empty and a special case in which the common part is the whole shape of both arguments.
There is an integer and a float add operation. Numeric arguments of different types are converted before the operation if possible.
The performance of this operation is excellent. There is little overhead. Basically just a loop and the add operation. The additional overhead needed for the third case is just an integer division in the loop.
Lets say the addition operator was a scalar operation between two scalar nouns, and a separate adverb or separate adverbs would handle the loop, then there would be a considerable overhead for each addition.Verify argument type and shape, select integer or float operator, check if type conversion is needed, perform type conversion, perform add. Now most of this is performed only once for the scalar operation over the whole arguments.
An advantage is that the notation can be used like the notation for matrix operations in mathematics.
A disadvantage is that the notation is different from that in most other languages.
It is also peculiar that the scalar operation verbs are said to have rank 0 0 and thus are supposed to operate on scalars, and still these operations operate on the whole of their left and right argument arrays with the help of what I call the scalar operation helper program.
It is strange with these helper programs which are never mentioned in the J documentation, but which must be understood by the user anyway, since they are obviously needed to explain how scalar operations can operate on whole array arguments.
As I see it the operation is complex. Seen on its own you can easily understand its function, but when used inside of the rank operator and inside other operators it is not easy to understand how the operators work together.
There is also an irregularity. Let’s call the third operation the Agree operation and the first the Map operation. There is another possible similar Agree operator with the shapes right-aligned. A row added to a table using this Agree operator would instead add the row argument to each row in the table argument. There is a possible general case in which the smaller argument dimensions could match any selected dimensions in the larger argument and the shorter shape argument would be logically replicated to the size of the larger shape argument.
Both the left-aligned and the right-aligned Agree operators are highly useful. The right-aligned Agree operator can be performed with the Rank adverb. The general case might be too complex to be a built-in operator. In JWithATwist there is a general transpose verb with which you can transpose the argument with the longer shape to a similar argument where the dimensions corresponding to the dimensions of the argument with the shorter shape is right- or left-aligned within the shape of the argument with the longer shape. You can then use the right- or left-aligned Align operators and transpose the result back. This means that you can easily write a code snippet which handles the general case, but that there would be some performance penalty compared to an operator for this general case.
A possible change would be to make the scalar operations scalar, make the helper programs explicit and take the performance penalty. There could be a Map adverb, and AlignLeft and AlignRight adverbs. These operations would then have to give domain error if the verb argument is not a scalar verb. The rank of the verbs would have to be registered.
Let’s say you have a chain of scalar operations. It would be more computationally efficient to make this chain into a single scalar verb than to execute a chain of Align- or Map- operations. It would also describe the algorithm you would use in most other languages better. To do this you would need the possibility to create user-defined scalar verbs.
If the scalar verbs had a type, the performance penalty could be avoided and the type verifications and type conversions could be moved to the Align- and Map- operations. The + verb would have to be split into a +Integer and a +Real verb in JWithATwist, with types like Integer -> Integer -> Integer and Real -> Real -> Real if we use the F# type notation.
As you can see these are complex changes and we would get a totally different language. For now I decided to keep the more APL- and J-like notation. However, I documented the helper programs clearly.