Banjo is an alternative to Lua or Javascript for game development that is suitable for ahead of time error checking and optimization, and IDE-supported refactoring. This is possible due to the introduction of a static type system, referential transparency, immutability, and the elimination of certain dynamic features.
// Functions
f(x) = x * 2
f = (x) -> x * 2
// Objects
x = { foo = bar, baz = quux }
// Lists
y = [foo, bar, baz, quux]
// Let
f(x) = (
y = x*x
) => y*y
// Single-line
f(x) = (y = x*x) => y*y
// Methods
obj = {
val = 5
obj.plus_val(x) = x + obj.val
}
bla = obj.val // bla == 5
blo = obj.plus_val(3) // blo == 8
// Operator definitions
true = {
(true || x) = true
(true && x) = x
}
false = {
(false || x) = x
(false && x) = false
}
Pros: Very powerful
Cons: Some checking must be left until runtime and it may not be so clear to a user what is guaranteed at compile time (or not)
degrees vs radians and other units of measure mixups can be a huge source of error. Good support for units would be a big win.
Current thought:
FFI: The FFI interface should really just parse C++ header files, or source files for whatever language is being interfaced to. With the clang wrappers out there, it should be feasible to figure out what is defined in each header and make that accessible in a sane way without requiring a bunch of wrappers to be written.
Recent experience programming games in lua reveal that it is all too easy to try and compare IEEE numbers incorrectly because they're some small fraction off of expectations. Plain Math.floor() isn't the right way to round to the nearest integer, you have to use Math.floor(0.5 + ...) which is hard to remember and error-prone.
There should be a sane way to compare numbers within a tolerance, or specify an integer comparison versus a floating point one.
For maximum specificity people wouldn't use standard operators but instead specify exactly what kind of math they want to use with each operation. e.g. z = int32.add(x,y). But this seems very wordy while z = x + y is somewhat ambiguous.
Languages with multiple number types seem to like to stick with the widest of any type used in the operation, but no wider. Adding two ints gives an int - even if there's information lost to overflow.
Languages like LISP don't allow overflow when using the standard operators, they'll promote the type to a larger size as necessary. This is actually very nice although there is a cost to performance it's not much worse than Lua in this regard. They probably provide functions to perform fixed-format operations as well.
I don't know the right solution for this, but if one could use the standard operators and get the right results it would be a huge win.
One disadvantage of using indentation for blocks (as opposed to braces) is that you don't get the benefit of highlighting the matching brace if you select the end one. It might be possible to fix this using some sort of indentation coloring scheme.
See for example EditBox
One thing I seem to do a LOT in Java, Lua, Javascript is copy local variables to a same-named field of a record. Some kind of sane syntax to make that easier would be really nice.
Current approach:
f(a,b) = {a, b}
By not supplying a value, the value is copied from scope as a method.
Also the opposite - extract variables from an object into local variables:
{a,b} = {a = 1, b = 2} // Set a=1, b=2
To rename:
{foo = a, bar = b} = {foo = 1, bar = 2} // Set a=1, b=2
I think it's a good idea to keep evaluation order in top-down left-to-right order; that is, when checking what the definition of something is, one need only look up and to the left, not to the right or downwards.
Obviously with loops, recursion, and function calls this doesn't always make sense.
Obvious counter-examples of this are python's x if y else z where y has to be checked first, and Haskell's x+y where x = 10, y=5 which has the variable definitions coming after their use.
One common exception to the left-to-right rule is assignment - the variable to assign to is put on the left side even though it is the last term to be used. This exception is worthwhile, though, as people are more likely to be "scanning" the left column for a definition than a use of something.
is a big feature for Lua and Unity projects. I think when designing libraries this is something to keep in mind. I think having a system that actually serializes code into the saved state during a restart would be ideal.
Current idea:
head/tail are nicer than first/rest because head/tail are both four characters, thus code can line up more nicely. size might win over length for its similar length, although len() has its appeal as well. fst/snd I think are too abbreviated, but I appreciate their brevity.
Instead of first/rest a syntax like x[0], x[1:] might make sense - these call get(0) and getFrom(0), and then no head/tail methods are needed - however, I think head/tail reads better than brackets.
Ran into this idea that Fortress has which is interesting - it requires that whitespace not be used in a way that contradicts operator precedence. So a+b * c might be an error because the visual grouping of a+b makes it look like that might happen first.
Something else from Fortress is that operators are partially ordered for precedence - that is, there are some pairs of operators which cannot have their precedence compared and must be wrapped in parentheses. This can help require parentheses in "strange" situations, like mixing boolean, bitwise, and arithmetic operators. Expressions like 3 & 2 * 7 are pretty confusing. I get the impression Fortress might report an error/warning in this case.
Good blog post on 21st century programming that showed how J's "insert" operation that "inserts" an operator between elements to compute a result is a bit more intuitive than fold. Also showing how it is hard to remember argument order for fold. Good food for thought for library design.
In Python when you reference a field in an object, and that object is a function in the object's class, it creates a "bound" method for you to call that injects that magical "self" parameter into the argument list. Conceptually this means that the "call" operation is unaware of objects and methods, it just operates on callable.
In javascript, on the other hand, methods are not bound by default. Instead, when a call is made, it does things differently for a method call - it sets the hidden "this" parameter based on the object the function was fetched from. So the "call" operation has to know whether a method is being called and, if so, what object it is called on.
Current thought:
A banjo program should conceptually be one that is run on every event, with the current state of all continuous values available (time plus saved state and time-dependent values) plus a description of the event that occurred.
The result of the program is a list of actions to take, a procedure, which will be interpreted by the runtime. The procedure should not contain any functions, just plain data, or possibly even just opaque structures not even inspectable by the application.
One puzzle on my mind is how to handle methods/fields that may be missing. Functions may want to inspect an object and see whether it has certain fields or not, and perhaps check whether it conforms to some interface or structure.
I see this pattern often enough in Javascript that I'd like to have a simple syntax for it. Perhaps x.?y' to mean(hasattr(x, 'y') => some(x.y)) || none. Perhapsx?.?ywould meanx && x.?y`.
For a complex projection like x.{y,z}fields missing in the projected object could be considered missing in the resulting object, so if you try to access them with out using ?. you'd get a compile time error.
There's this slightly annoying repetition when one wants to use the same expression twice... you declare it and use it twice. In theory one could declare it as part of the first use. However, this would decrease readability. Something to ponder, perhaps:
a = b + c
k = a + 1
j = a / 2
vs
k = (a = b + c) + 1
j = a / 2
Doctest is kind of a cool way of writing little unit tests for things while documenting them at the same time. I think this use case is worth supporting in some way.
Current approach - use a magic local variable to specify some test cases, each of which must evaluate to a truthy value.
(Inspired by Elm's .foo syntax to create a field accessor function)
It would be very useful to support a method call whose target hasn't been specified yet, as a kind of currying. It should support acting as a getter or a setter, so maybe:
.y = {
(x) = x.y
lazy(x) = -> x.y
setter(x) = {(val) = x {+} {y = val}}}
mapper(f) = {(x) = x {+} {y = f(x.y)}}
}
.foo(bar) = [
(x) = x.foo(bar)
lazy(x) = -> x.foo(bar)
setter(x) = ... // ???
}
The setter is useful for higher-order functions where you want to create a function that replaces a field in its parameter with new value.
The mapper transforms the single field to give a new object where just that field was replaced.
When operating on rows one can add, remove, and update fields. A good syntax for this can be tricky to come up with.
I'm thinking that for object extension I can use an operator like @, ++, {+}, ɸ, or Ͼ.
Another good option would be to have an alternative syntax within { }'s, like {x + y} to concatenate two objects or {x + {...}} to extend an object with more fields. The interpretation of the curly brackets would thus depend on the operator inside.
The system needs to know which modules are entry points that take the system provided modules and return an automaton and which ones are modules that are meant to be used as a library.
My latest brain wave here is to have a special folder that identifies the root of an "application" called .banjo. Tools can use this as a marker for finding the root of the application, just like how git uses .git.
Files in the root folder are assigned to "global" variables for that project. The contents of the file is the value of the variable, parsed according to its extension. For *.banjo that means a banjo expression. For *.txt it could be a string. For *.html or *.json or *.jpg there could be some sensible wrapping process making these types of data useful in the app. Folders are translated into an object where each file in the folder is a slot of the object.
Where there are two files with the same base name on the path, the values are concatenated with @ in search path order, then folders before files, and then alphabetical order (by suffix).
In this way the entire project can be combined into one big AST.
Importing libraries can be done by putting a banjo source file that has an expression that loads and returns the library. For example, a file somelib.banjo could contain the expression lib.link library({name = 'somelib-1.0'}) and lib.link library would presumably do whatever magic is needed to get that library as an object. A library would not share the same globals as a project it was imported to, but its calls to lib.link library would typically pull from the same place. This is similar to how python does things with its .pth files.
Literals can be translated into calls to certain functions in the scope, in which case there would need to be at least one predictable fixed path to find those literals.
A banjo environment should provide a tool that creates and manages the .banjo folder (which contains caches used by the compiler and dependency manager) and at the same time installs the core libraries into the project at project creation time. It can also help to search for and install/list/uninstall dependencies.
Having created a project, a tool is needed to evaluate expressions within that project - this is how a project can be "run". Or a REPL could be used. Different tools will interpret the result of the expression differently, whether to "build" or "run" or "print" something depends on the tool used.
When no .banjo folder can be found in the same folder as a source file or its parent, only the "default" core globals are provided.
The "main functions" are detected by evaluating each top-level variable and seeing whether it returns the appropriate type of result - the return of a call to main or whatever this function is eventually called.
Similarly, exported libraries can be detected by looking at their evaluated value as well, to see whether they are computing a library value returned by, perhaps, a function named library.
It might be interesting to compare the Haskell monad approach to a fluent interface:
do putStr "Enter your name: "
name <- readString
putStr "Hello, " ++ name
Possible similar code using method calls in Banjo:
io.putStr("Enter your name: ")
.readString.if({
string(io, name) = io.putStr("Hello, " + name)
error(io) = io.quit
})
Note the use of the visitor pattern on the readString result to pass in the success/failure of the operation. This is visually a bit messier than throwing an exception but it does make for very clear error handling.
I think it may be a good idea to disallow variable shadowing, or at least warn about it.
I think dependencies on libraries should be very specific in the version they use.
Dependencies of dependencies are still a bit of a trick, though. Should they be shared? What if you want to provide a customized / mocked version of a library to a library you use?
A variadic function may accept a varying number of arguments. A language that supports variadic functions typically provides a list or array or iterator style interface to the arguments.
Varargs is a necessity if you want to write a delegating wrapper function that can take all its arguments and pass them along to some other function, possibly modifying them in some way.
I think the way to go here isn't new syntax - just have an intrinsic function that takes a function accepting a list of arguments and returns a function that puts all its arguments into a list and passes it through to the original one.
And for the reverse, provide a function that takes a function and a list of arguments and calls the function with that list. So:
varargs(f) = (...) -> f([...])
apply(f, args) = f(args expanded)
These functions cannot be written in the language itself.
In Scala and some other functional languages you can create a lambda using placeholder syntax, like (_ + _) means something like (_1,_2) -> _1 + _2. Elixir uses syntax like &(&2 + &1) for a quick lambda or &foo.bar to create a bound method.
This seems handy for very small functions like this one, where you are basically creating a function from a simple expression.
I do wonder whether we're buying much here, given you're basically saving one character from x -> x + 1 vs -> $1 + 1. And the version with a named parameter could have a meaningful named parameter, too. BUT if we're relying on these quick lambdas to support currying and bound methods, it might be worth it. Or if the syntax can be shortened even more.
It seems like these would almost always need to be wrapped in parentheses so perhaps a variation on parentheses would be good like &(&2.bla(&1)) means _1,_2->_2.bla(_1)
One way to set up the system is to use the Actor model at the core, and implement the FRP on top of that. The Signal objects would implement the actor interface, and having a "main" that returned a Signal would thus be returning an actor suitable for use by the runtime. This would be pretty flexible in allowing other non-FRP type applications as well.
I think a cool idea is to apply optimization in a manner like CSS. The programmer provides hints, strategies, and assumptions based on a selector that supplies to the call graph. So one can suggest a certain list be represented using an array that is copy on write rather than a singly linked list. Or that a number is usually small and can be a native number of some sort most of the time. The cod generator can use this to produce more efficient code.
The optimization information can't be dumped right into the implementation code because it varies depending on how that code is called.
Being CSS like means the selectors can be a bit fuzzy to provide defaults when the use hadn't optimized yet.
A profiler could suggest annotations for a particular application based on data collection.
The target platform should also be part of the selector so that different platforms can have different options.
Number, String, List, ...list like, string likeList.with object, String.from code pointsnumber, string, data, 'strings,players`, ...I've been mulling over this problem that sometimes the host language might provide UTF-16 strings (Java or Javascript) or UTF-8 strings (C) but you'd sometimes want to write code that runs efficiently in both environments.
I think that since ASCII is a subset of both and many (most?) operations are only interested in the ASCII characters of the string, providing an API that handles ASCII specially could be implemented efficiently in both languages. For example:
Code that does support or care about unicode will inevitably want APIs that operate on code points, which requires (inefficient) translation regardless of whether the underlying string is UTF-8 or UTF-16.
I think the key thing here is to provide operations for as many of the things people want to do while avoiding having them think about or depend on the underlying encoding. Operations like length() and indexOf() are troublesome because they are ambiguous about whether code points or they correspond to (8 or 16 bit) code units or (32 bit) code points. So providing useful operations that mostly eliminate the need to ask about the length is good. Like, we never want people doing a loop using a length and index...
At first I thought the super call could be something simple like super(x) because, hey, you know what method you want to call (the one you are in) and what to call it on (self). But, I was forgetting that you might be in a method inside a method inside a method. So just saying super wouldn't allow you to specify which super in the current lexical scope you are referring to.
Now I'm thinking self:method(args) to invoke a super method, giving the method name is redundant but not terrible, and the redundancy may help avoid some copy/paste errors at times.
I was mulling over how to support mutually recursive functions in a nice way using let. A simply recursive function is passed itself automatically when it is called as part of the standard calling convention. However, in a situation like:
(
f1(x) = ... uses f2 ...
f2(x) = ... uses f1 ...
) => ... uses f1 and/or f2 ...
The functions don't immediately have any access to each other.
However, for this special case I realized that I could inline the definitions of functions in the same let block, but which come later, into the definition of a function.
So, in this example f1 wouldn't normally have access to f2 because it isn't defined yet. However, I can copy the definition of f2 into the body of f1 if necessary. This might require an analysis to decide which functions need to be inlined.
So the above could be translated to:
(
f1(x) = (f2(x) = ... uses f1 ...) => ... uses f2 ...
f2(x) = ... uses f1 ...
) => ... uses f1 and/or f2 ...
It has become clear from using Java that even if there are errors detected in the code, one should allow the program to run nonetheless. The piece of code that has the error may not be part of what the programmer is trying to test. This allows the programmer to build their program incrementally and leave areas unimplemented that are truly unimplemented rather than having to stub things out.
Of course, if the code with errors is encountered, the program should exit with as helpful an error as possible.
And, the IDE or toolchain should be reporting these errors while still allowing the program to be run as much as possible.
.?The .? operator allows one to catch a method that is undefined or that fails a precondition, and in so doing possibly support some kind of overloading / duck typing.
If we treat methods that are not defined as methods that always fail a precondition, we can simply say that x.?foo returns [x.foo] when x.foo passes all preconditions, and [] otherwise.
A postcondition failure is another matter, though ...
The system needs some sort of bootstrap that provides default versions of various builtins such as literal values and the ability to import modules.
However, it would be nice if it were possible to load a module using a custom environment that defines "pimped" versions of literals like string and number.
Object Algebras are a pretty interesting library structuring technique that provides maximum extensibility. I think this approach can be used at least for the AST for the language itself and possibly in many other places in the library.
I did some reading today about Agda, a dependently-typed language. I think there are some neat ideas in there.
For any declared contract requirement there are some contracts that could be checked statically and this should be done as much as possible in the programming system. This is going to be a similar job as the dependent type checker in Agda is doing.
Also it should be possible to determine statically in most cases when someone is trying to call a method that is not or may not be defined.
And, totality checking also is interesting and it would be nice to give an error if something is being called in a way that may run forever.
It would be very nice to have, for example, the ability to determine whether a list is empty or non-empty statically and allow the programmer to call a method to get the first element and know that they'll get an error from the analyser if they may have made a mistake.
One possible inconvience in the system is that I/O operations that seem like "just reads" without side-effects actually must have a side-effect of requesting the desired data.
Two ideas I've had so far to help avoid having to return back all these side effects from code that just does I/O reads are:
Both are equivalent in a way, actually.
It sounds strange, but I think it's helpful to provide a set representing "all objects" and then make subsets from that for classes. For example
objects like { length ∈ natural numbers }
Should be a set that is seemingly enumerable.
However, at runtime if someone actually enumerates that set we might be in trouble - we don't really want to provide an implementation of that. So that's where the whole "strategies" idea comes in.
Recently I have been pondering how to propagate contract failures or not. I want this to be controlled - a contract failure shouldn't propagate accidentally.
So I have devised a system as such, firstly to "catch" a contract failure, use .? with ||:
x.?foo(1) || not accepted
Methods can define contracts as part of the signature, this is translated into a precondition of the method, notated using &&&:
{ foo(n) = (n != 1) &&& (n) }
To propagate a precondition failure, as one will often want to do in a higher-order function, you can use a precondition:
{ (x # y) = y.?foo &&& y.foo
The expression is repeat here, in a way, but the system should be able to optimize this to be free.
This leads to a definition of the "variant" syntax feature where #foo = { (x # y) = y.?foo &&& y.foo and thus (#foo #? {}) || fallback) == fallback, now we can provide the "default" case for a visitor.
This algorithm might be the one to look at when implementing the type-checker:
http://matt.might.net/articles/implementation-of-kcfa-and-0cfa/
Laziness is required for certain things:
true = lib.booleans.true and false = lib.booleans.false lazily then even the definitions of lib, lib.booleans, lib.booleans.true, and lib.booleans.false can use true and false. With call-by-need this is OK but it may not terminate in call-by-value or call-by-name scenarios.It's helpful to keep this in mind when designing the library for storage. It should be relatively easy to manage the three stages of data.
It's interesting at the moment to see a kind of concatenative programming evolve where if objects are callable by passing themselves or their relevant contents to a provided function, then you can kind of chain values along.
Interesting ideas here for the UI library:
Interally the compiler has some kinds of "macros" to support some of the internally implemented operators. How to make these macros hygienic in the sense that they can create new local variables that cannot conflict with local variables from the code they are wrapping has been puzzling me for some time.
One idea I have had recently is to namespace all the identifiers, not unlike how LISP does it. Method names would all have to be in the same namespace, by default.
In this way, a macro can generate some code without fear of creating variable conflicts.
The default namespace for a file could be its filename.
In order to serialize / de-serialize expressions to and from source there would need to be a way to denote the namespace of an identifier for cases where the macro has already been expanded.
I haven't figured out what the syntax for this should be, but I think it shouldn't be too complex. Perhaps if a '/' is embedded in the identifier it would assume that the identifier is fully qualified, otherwise the identifier would be resolved relative to the current namespace, but if there is a / the namespace is left as-is. When printing source we can strip off anything before the / if it matches the current namespace prefix.
e.g.
foo == current namespace\/foo
x\/foo --> x\/foo
http://www.gnu.org/software/kawa/internals/numbers.html
Kawa is a scheme dialect with the Scheme numerical tower enhanced with support for units of measure.
For binary operators, Kawa calls a method of the left-hand object. That object tries to see if it has been given an object type it recognises; if not, it will fallback to calling a "reversed" version on the right-hand side. The "reversed" version should not fall back yet again.
Facebook has been teaching about the Flux architecture it uses for web apps, which is kind of FRP-based. Some useful ideas there for how applications / games can be structured.
One idea I've had for debugging in a pure language would be to back-trace the computation of a value.
For example if you're looking in the reactor at a value, while debugging it could be annotated with the actual function call that produced it. So you could expand that value into a function call ... and the look at its arguments and possibly expand one or more of them to work your back to where there might have been a problem. It wouldn't have to run in this mode all the time ... it should be possible to "re-run" the last frame in this mode to yield these annotated values instead of the usual vanilla values.
In Scala, types often change the meaning of something. For example, if you assign a expression into a lazy method parameter it's automatically made into a thunk. If you call a method that takes no parameters, the parens are optional. I think this is a bad idea ... it's extra complexity.
In Haskell, too, the static type environment is used to resolve type-class invokations. This includes the expected target type of the expression. I think this is a lot of complexity to consider when reading code - you have to not only know the type of an expression, but also the type expected of that expression by whatever it is being passed to.
In Banjo I want to keep it relatively simple. There's no overloading based on static parameter types or return types, only the actual runtime value matters for resolving a slot value, just like in Javascript and other dynamic languages.
String interpolation seems like a fun language feature - that is, having a syntax like "I see ${n} apples" where a local variable n is substituted into the string at the appropriate place.
However, I think this makes localization more difficult. The python approach of putting the substitutions afterwards is probably better.