Java/Scala/Prolog integration and cross-platform Scala

1. Why combine Java, Scala and Prolog?

The proposed scenario: Java or Scala for the aspects "in the large" and the interaction with the operating system, Prolog for some data and behaviour of the core. Examples: modelling the mind of an agent in Prolog; writing the computer's strategy of a game in Prolog; storing data and knowledge in Prolog databases.

2. Which kinds of integration are there?

When language A uses language B, the integration can be:

• library-oriented: a library of A gives access to B. Handy, but the glue code can get complicated;
• language-oriented: A is extended with the abstractions of B. The resulting language is often too complicated;
• encapsulation-oriented: code in B is wrapped into abstractions of A. A good trade-off between the first two;
• DSL-embedding, the modern approach, not used by tuProlog: B is used through a DSL that is actually an API of A. Another good trade-off.

The slides' outline writes the first predicate as ->/2; the operator tuProlog defines, and the one the examples use, is <-.

3. How does Prolog call Java?

This Java program opens a window with the javax.swing library:

public class C {
  public static void main(String[] s) {
    javax.swing.JFrame f = new javax.swing.JFrame();
    f.setSize(100, 100);
    f.setVisible(true);
  }
}

The same in Prolog, run with ?- createframe('Prova', 400, 300).:

createframe(Name, X, Y) :-
    new_object('javax.swing.JFrame', [], Obj),
    Obj <- setTitle(Name),
    Obj <- setSize(X, Y),
    Obj <- setVisible(true).

• new_object(+ClassName, +ArgumentList, -ObjRef) calls the constructor of the class, found through the classpath of the tuProlog application by its package-qualified name. The arguments can be integers, strings or Java objects. ObjRef becomes a Prolog term that stands for the object: ?- new_object('java.lang.Integer', [1], X). gives X = '$obj_0'.
• ObjRef <- MethName(Args) calls a method when no result is needed.
• ObjRef <- MethName(Args) returns ObjRef2 unifies ObjRef2 with the result: after setVisible(true), Obj <- isVisible returns Z gives Z = true.
• Other predicates read and write fields, create and access Java arrays, load classes dynamically, and so on (see the tuProlog documentation).

A typical use is a small Prolog library over Java objects, here fast arrays of integers backed by java.util.Vector. The slides list vector_new(-V), vector_add(+V,+E), vector_addall(+V,+V2), vector_clear(+V), vector_contains(+V,+E), vector_lookup(+V,+Pos,-E), vector_isempty(+V), vector_size(+V,-S), vector_print(+V), and implement four:

vector_new(V) :- new_object('java.util.Vector', [], V).
vector_add(V, E) :- new_object('java.lang.Integer', [E], I), V <- add(I).
vector_size(V, S) :- V <- size returns S.
vector_lookup(V, P, E) :- V <- elementAt(P) returns E.

?- vector_new(X), vector_add(X,1), vector_add(X,2), vector_lookup(X,0,N).   % X = '$obj_0', N = 1
?- vector_new(X), vector_add(X,1), vector_add(X,2), vector_size(X,S).       % X = '$obj_0', S = 2

4. How does Java call Prolog?

The tuProlog library (package alice.tuprolog) offers classes for an engine, a term and a solution, and methods to solve goals, ask for further solutions and navigate terms:

• Prolog: a "Prolog virtual machine", the engine;
• Theory: a Prolog database, given as text or as a list of clauses;
• SolveInfo: the outcome of a goal;
• Term: the root of the term hierarchy. Its subclasses are Var (a variable, named or anonymous), Number (Int, Long, Float, Double) and Struct (a compound term, and also a list). Among its methods: getRenamedCopy (renames the variables), isAtom (a constant name or number?), isList, unify (does this unify with that?).

The slides' five examples (package u12 and the throws Exception clauses as in the slides), with the output they print:

// libraryDependencies += "it.unibo.alice.tuprolog" % "tuprolog" % "3.3.0"
import alice.tuprolog.*;

public class Test1 {                       // direct goal solution
  public static void main(String[] args) throws Exception {
    final Prolog engine = new Prolog();
    final SolveInfo info = engine.solve("append([1],[2,3],X).");
    System.out.println(info.getSolution());
    // append([1],[2,3],[1,2,3])
  }
}

public class Test2 {                       // getting all solutions
  public static void main(String[] args) throws Exception {
    final Prolog engine = new Prolog();
    SolveInfo info = engine.solve("append(X,Y,[1,2,3]).");
    while (info.isSuccess()) {
      System.out.print("solution: " + info.getSolution() + " - ");
      System.out.println("bindings: X/" + info.getTerm("X") + " Y/" + info.getTerm("Y"));
      if (engine.hasOpenAlternatives()) {
        info = engine.solveNext();
      } else {
        break;
      }
    }
    // solution: append([],[1,2,3],[1,2,3]) - bindings: X/[] Y/[1,2,3]
    // solution: append([1],[2,3],[1,2,3]) - bindings: X/[1] Y/[2,3]
    // solution: append([1,2],[3],[1,2,3]) - bindings: X/[1,2] Y/[3]
    // solution: append([1,2,3],[],[1,2,3]) - bindings: X/[1,2,3] Y/[]
  }
}

public class Test3 {                       // creating and using terms
  public static void main(String[] args) throws Exception {
    final Prolog engine = new Prolog();
    final Term list1 = engine.toTerm("[1]");                              // [1]
    final Term list2 = new Struct(new Term[]{new Int(2), new Int(3)});    // [2,3]
    final Term app = new Struct("append", list1, list2, new Var("X"));    // append([1],[2,3],X)
    final SolveInfo info = engine.solve(app);
    System.out.println(info.getSolution());
    // append([1],[2,3],[1,2,3])
  }
}

public class Test4 {                       // reading terms
  public static void main(String[] args) throws Exception {
    final Prolog engine = new Prolog();
    Term t = new Struct();                          // []
    for (int i = 10; i >= 0; i--) {
      t = new Struct(new Int(i), t);                // t will be [0,1,2,...,9,10]
    }
    final Term app = new Struct("append", t, t, new Var("X"));
    final SolveInfo info = engine.solve(app);
    final Struct t2 = (Struct) info.getTerm("X");
    for (java.util.Iterator<? extends Term> i = t2.listIterator(); i.hasNext();) {
      System.out.print(" " + i.next());
    }
    // 0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
  }
}

public class Test5 {                       // setting a theory
  public static void main(String[] args) throws Exception {
    final Prolog engine = new Prolog();
    final Theory t = new Theory(
        "search(E,[E|_]). " +               // a space is needed here!
        "search(E,[_|L]) :- search(E,L).");
    // engine.setTheory(new Theory(new FileInputStream("file.pl")));
    engine.setTheory(t);
    final SolveInfo info = engine.solve("search(1,[1,2,3]).");
    System.out.println(" " + info.getSolution());
    // search(1,[1,2,3])
  }
}

The space matters because a clause ends with a period followed by white space; without it tuProlog rejects the theory. Test4's iterator is typed here as Iterator<? extends Term>; the slide uses a raw Iterator, which also compiles.

When is it worth it? Whenever something is better implemented in Prolog. The slides' exercise: permutations are a one-line idea in Prolog, perm(+L, -L2); write a Java method that takes a LinkedList and returns an Iterator<LinkedList>, whose next() calls solveNext() and builds the Java list from the resulting term.

5. How does a Java class become a Prolog library?

A Prolog library is basically a Prolog database that defines many predicates. It is loaded from a theory with the goal load_library/1, or in the IDE with the library manager. The IDE loads the built-in ones automatically: BasicLibrary (standard predicates such as append), ISOLibrary (the predicates of the ISO standard), IOLibrary (input/output, e.g. tell, told).

Writing one in Java is an elegant, encapsulation-oriented approach. Write a class in which each method defines a predicate, compile it (say as lib.mylibrary) and load it with load_library('lib.mylibrary'). The method's name and signature decide what it defines:

• public boolean <pred_name>_<N>(T1 arg1, …, TN argN) defines the predicate pred_name/N, which succeeds when the method returns true;
• public Term <eval_name>_<N>(T1 arg1, …, TN argN) defines an evaluable function, usable on the right of is/2;
• each Ti is Term, Var, and so on.

// run with: java -cp <path-to-libraries>:2p.jar alice.tuprologx.ide.GUILauncher
// be sure the library is compiled with Java 8
import alice.tuprolog.*;
import alice.tuprolog.Number;   // not in the slides: without it, Number is ambiguous with java.lang.Number

public class TestLibrary extends Library {

  private int termAsInt(Term t) {
    return ((Number) t).intValue();
  }

  // try with: X is sum(10,20).
  public Term sum_2(Term arg0, Term arg1) {
    return new Int(termAsInt(arg0) + termAsInt(arg1));
  }

  // try with: X is minus(20,10).
  public Term minus_2(Term arg0, Term arg1) {
    return new Int(termAsInt(arg0) - termAsInt(arg1));
  }

  // try with: sum(Y,8,21).
  public boolean sum_3(Term arg0, Term arg1, Term arg) {
    try {
      if (arg instanceof Var) {
        return arg.unify(this.getEngine(), sum_2(arg0, arg1));
      }
      if (arg0 instanceof Var) {
        return arg0.unify(this.getEngine(), minus_2(arg, arg1));
      }
      return arg1.unify(this.getEngine(), minus_2(arg, arg0));
    } catch (Exception e) {
      return false;
    }
  }
}

?- X is sum(10,20).    % X = 30
?- X is minus(20,10).  % X = 10
?- sum(Y,8,21).        % Y = 13
?- sum(10,8,Z).        % Z = 18
?- sum(10,Y,21).       % Y = 11
?- sum(10,8,19).       % no

sum_3 is relational in the Java way: it looks at which argument is a variable and asks the engine to unify it with the value computed from the other two. arg.unify(getEngine(), term) returns whether the unification succeeded, and that is the success of the predicate. The last query is the "check" mode: Z is not a variable, so the method unifies 8 with 19 − 10 = 9, which fails.

6. How does Scala wrap a Prolog theory?

The encapsulation-oriented approach on the Scala side: a theory becomes a Scala function from goals to lazy streams of solutions.

// with dependency: "it.unibo.alice.tuprolog" % "tuprolog" % "3.3.0"
import alice.tuprolog.*

object Scala2P:
  given Conversion[String, Term] = Term.createTerm(_)
  given Conversion[Seq[_], Term] = _.mkString("[", ",", "]")

  def extractTerm(t: Term, i: Integer): Term =
    t.asInstanceOf[Struct].getArg(i).getTerm

  def mkPrologEngine(clauses: String*): Term => LazyList[Term] =
    val engine = Prolog()
    engine.setTheory(Theory(clauses mkString " "))
    goal => new Iterable[Term]{
      override def iterator = new Iterator[Term]{
        var solution = engine.solve(goal);
        override def hasNext =
          solution.isSuccess || solution.hasOpenAlternatives
        override def next() =
          try { solution.getSolution } finally { solution = engine.solveNext }
      }
    }.to(LazyList)

object TryScala2P extends App:
  import Scala2P.{*, given}

  val engine: Term => LazyList[Term] = mkPrologEngine("""
    member([H|T],H,T).
    member([H|T],E,[H|T2]):- member(T,E,T2).
    permutation([],[]).
    permutation(L,[H|TP]) :- member(L,H,T), permutation(T,TP).
  """)

  engine("permutation([1,2,3],L)") foreach (println(_))
  // permutation([1,2,3],[1,2,3]) ... permutation([1,2,3],[3,2,1])   (six lines)

  val input = Struct("permutation", (1 to 20), Var())
  engine(input) map (extractTerm(_, 1)) foreach (println(_))
  // [1,2,3,4,...,20]  [1,2,...,18,20,19]  [1,2,...,17,19,18,20]  ... (20! of them, computed on demand)

How the pieces fit:

• The two given Conversions let a String stand for the term it parses to, and a Scala Seq for a Prolog list, through its text [a,b,c] and then the first conversion. So engine("permutation([1,2,3],L)") and Struct("permutation", (1 to 20), Var()) both typecheck. The second goal is built structurally, with Var() a fresh anonymous variable.
• mkPrologEngine creates one engine with the clauses as its theory and returns a function. For each goal, an Iterator calls solve once and, after handing out each solution in next(), advances with solveNext.
• Turned into a LazyList, the solutions are computed only when the stream is consumed. Prolog's backtracking becomes a monadic stream you can map, filter or use in a for-comprehension. extractTerm(t, i) reads argument i of a solution, here the permutation.

    goal =>
      LazyList.unfold(Option(engine.solve(goal)).filter(_.isSuccess)): info =>
        info.map(i => (i.getSolution,
          Option.when(engine.hasOpenAlternatives)(engine.solveNext).filter(_.isSuccess)))

// engine("append([1],[2],X)").toList   ==  List(append([1],[2],[1,2]))
// engine("member([1,2],E,R)").toList   ==  List(member([1,2],1,[2]), member([1,2],2,[1]))

A preview: logic programming in plain Scala

The same search can be written without an engine. A fully relational predicate becomes a function returning the LazyList of all its solutions, and the "mode" is chosen afterwards by filtering the tuples:

// lookup([H|T], H, zero, T).
// lookup([H|T], E, s(N), [H|T2]) :- lookup(T, E, N, T2).

def lookup[A](list: List[A]): LazyList[(A, Int, List[A])] = list match
  case Nil => LazyList()
  case h :: t =>
    (h, 0, t) #::
      (for (e, n, t2) <- lookup(t) yield (e, n + 1, h :: t2))

println(lookup(List(10, 20, 30, 20)).toList)
// List((10,0,List(20, 30, 20)), (20,1,List(10, 30, 20)), (30,2,List(10, 20, 20)), (20,3,List(10, 20, 30)))
println(lookup(List(10, 20, 30, 20)).collect { case (20, n, _) => n }.toList)   // List(1, 3)
println(lookup(List(10, 20, 30, 40)).collect { case (e, 3, _) => e }.toList)    // List(40)

The slide's output comments show positions as zero, s(zero), … as in the Prolog version; the Scala code uses Int, and prints 0, 1, 3.

init(s(0,0)).                                  % initial condition
goal(s(3,3)).                                  % goal
move(up, s(X,Y), s(X2,Y)) :- X > 0, X2 is X-1.
move(down, s(X,Y), s(X2,Y)) :- X < 3, X2 is X+1.
move(left, s(X,Y), s(X,Y2)) :- Y > 0, Y2 is Y-1.
move(right, s(X,Y), s(X,Y2)) :- Y < 3, Y2 is Y+1.

% one possible plan/2 (not from the slides)
plan(MaxN, Trace) :- init(S), plan(S, MaxN, Trace).
plan(S, _, []) :- goal(S).
plan(S, N, [D|T]) :- N > 0, move(D, S, S2), N2 is N-1, plan(S2, N2, T).

?- plan(6, T).    % T = [down,down,down,right,right,right] ; T = [down,down,right,down,right,right] ; ...  (20 plans)
?- plan(5, T).    % no

7. How does Scala run outside the JVM?

(Guest talk by G. Aguzzi.) Cross-platform software, also called platform-agnostic or multi-platform, runs on several target platforms (operating systems, architectures, runtimes) from one codebase. The motivations: lower development costs, no duplicated code, and business logic that stays in sync across platforms. The classical model is the JVM's "write once, run everywhere" with portable bytecode. The modern shift is target compilation: transpilation from one high-level language to another (Scala, Kotlin or Dart to JavaScript), or ahead-of-time compilation to machine code through toolchains such as LLVM (Scala Native, Rust, Go). Browser JIT engines and WebAssembly (Wasm) runtimes are the new portable environments, while UI SDKs stay platform-specific and share a core written in one language.

For Scala, the benefits are a shared codebase (no duplication, no error propagation), full-stack development with one language for backend and client, and access to platform libraries that were not written for Scala (TensorFlow from Scala). The use cases: cross-platform libraries (e.g. Cats), web applications (Scala.js), cloud lambda functions or containers (native), code shared by Android and iOS apps. Compiling target code to Scala is usually not considered.

Configuring and compiling

scalac does not produce JavaScript or native executables by itself: SBT plugins do it. Plugins go in project/plugins.sbt, the projects in build.sbt:

// project/plugins.sbt
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.21.0")
// for native
addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.5.10")

// build.sbt
import sbt.Keys.scalaVersion
ThisBuild / scalaVersion := "3.3.5"
name := "scala-js-native"
// multi-project configuration: each val is a subproject with that name
val js = project.settings(
  scalaJSUseMainModuleInitializer := true
).enablePlugins(ScalaJSPlugin)
val native = project
  .settings(
    nativeConfig ~= { _.withBaseName("native-out") }
  )
  .enablePlugins(ScalaNativePlugin)

How it works (from S. Doeraene's thesis, Cross-Platform Language Design, EPFL 2018): a compiler plugin turns plain Scala into an intermediate representation (IR) that contains the platform-dependent aspects; from the IR the toolchain optimises, links and manages dependencies. Scala 3 supports both targets out of the box, and givens, extension methods, union types and enums compile to JS and native code without performance penalties. The Metals IDE switches between the JVM, JS and native environments, and TASTy (typed abstract syntax trees) improves the binary compatibility of libraries across targets. Since Scala.js 1.17 an experimental WebAssembly target uses the WasmGC and exception-handling proposals: faster compute-heavy code and less JIT warm-up in browsers and Node.js.

// build.sbt: enabling Wasm
scalaJSLinkerConfig := {
  scalaJSLinkerConfig.value
    .withExperimentalUseWebAssembly(true)
    .withModuleKind(ModuleKind.ESModule)
}

Limits, and how mature each target is

A pure cross-platform project cannot use the JVM ecosystem (threads, file systems) unless a cross-platform facade or a reimplemented library exists. Output bundles include parts of the Scala runtime and library: a Scala.js bundle easily reaches about 1 MB before tree-shaking and minification.

8. How does Scala talk to JavaScript and C?

Scala code needs type declarations that bridge Scala types to JS globals or C headers. In Scala.js they are facades: ordinary Scala objects or classes annotated to tell the compiler that they stand for JavaScript APIs. In Scala Native they are bindings to C libraries, with @extern and low-level pointers (Ptr[Byte]).

// Scala.js facade (browser alert)
import scala.scalajs.js
import scala.scalajs.js.annotation.*

@js.native
@JSGlobal
object window extends js.Object {
  def alert(msg: String): Unit = js.native
}

// Scala Native C binding (math)
import scala.scalanative.unsafe.*

@link("m")
@extern
object libm {
  def sqrt(d: Double): Double = extern
}

The slides omit the imports, added here; with them the facade compiles with Scala.js 1.22 (the binding was not compiled: no LLVM toolchain was available). Scala.js interoperability rests on three pillars:

• facades (@js.native): pure type definitions of existing JS APIs. They generate no JS code, and no method bodies are emitted; they tell the compiler how to emit the right calls. js.Object is the base of every type that represents a native JS object, @JSImport("@tensorflow/tfjs", JSImport.Namespace) tells the linker to import an NPM package or ES module, and js.UndefOr[A] is a value that may be undefined, like an Option;
• JS conversions (scala.scalajs.js.JSConverters.*): between Scala collections (Seq, Map, Option) and JS ones (js.Array, js.Dictionary, js.UndefOr), e.g. seq.toJSArray, and from JS promises to Scala futures with .toFuture;
• exports (@JSExportTopLevel): Scala classes and objects exposed to JavaScript code, so that calls go both ways.

Writing facades by hand for large libraries (React, Three.js, Lodash) is practically impossible. But TypeScript's DefinitelyTyped repository has .d.ts type definitions for almost every NPM package, and ScalablyTyped, a compiler and SBT plugin, turns them into type-safe, idiomatic Scala.js facades. TypeScript modules become Scala packages, and union types, structural types and callbacks become Scala 3 union types, generics and functions, checked at compile time. The whole NPM ecosystem becomes usable with type safety and IDE completion. A clean wrapper then hides from its clients the interop annotations, the collection conversions and low-level details such as calling dispose() on tensors:

import scala.scalajs.js
import customtypings.tensorflowTfjs.mod as tf

val data = js.Array(1.0, 2.0, 3.0, 4.0)
val tensor = tf.tensor1d(data)
val squared = tensor.square()
tensor.dispose()

lazy val scalablytyped = (project in file("scalablytyped"))
  .enablePlugins(ScalaJSBundlerPlugin, ScalablyTypedConverterGenSourcePlugin)
  .settings(name := "scala-js-facade-scalablytyped",
    scalaJSUseMainModuleInitializer := true,
    Compile / npmDependencies += "@tensorflow/tfjs" -> "4.22.0",
    stOutputPackage := "customtypings")

Raw facades are rarely needed for the UI: Laminar is a pure-Scala reactive UI library based on observables, which targets the DOM directly without JS dependencies; Tyrian, from the Typelevel ecosystem, follows the Elm architecture (model-view-update) and integrates with Cats Effect.

One project, three targets

The sbt-crossproject plugin sets up multi-platform projects with a standard layout. CrossType.Pure is for pure cross-platform projects such as libraries, with all the code under src/. CrossType.Full is for projects with platform-specific code: shared holds the common code (data structures, interfaces, …), and js, jvm and native the platform code (library usage, GUIs, …). Dependencies use %%% so that each platform gets its own artifact.

// project/plugins.sbt
val crossOrg = "org.portable-scala"
val crossNative = "sbt-scala-native-crossproject"
val crossJs = "sbt-scalajs-crossproject"
addSbtPlugin(crossOrg % crossJs % "1.3.2")
addSbtPlugin(crossOrg % crossNative % "1.3.2")
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.21.0")
addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.5.10")

// build.sbt
crossProject(JSPlatform, NativePlatform, JVMPlatform)
  .crossType(CrossType.Full)            // or CrossType.Pure
  // common settings, use %%% for cross libraries
  .settings(
    libraryDependencies ++= Seq(
      "org.scalatest" %%% "scalatest" % "3.2.19"
    )
  )
  .jsSettings()
  .nativeSettings()
  .jvmSettings()                        // standard "scala" world

The architecture that fits is clean (hexagonal) architecture, ports and adapters. The pure core, in the shared project, holds domain models, business logic, validation rules and pure functions, and uses only cross-platform libraries (e.g. Cats, Circe). Platform adapters implement the ports per target: database engines and server frameworks on the JVM (JDBC, Doobie), DOM and browser storage in JS (Laminar, Tyrian), direct memory, POSIX files and C interop in native code. Generic traits are defined in the shared module and implemented on each platform; SBT handles cross-linking and packaging.

Most major Scala libraries are published for all three targets at once, so business logic, data validation and test suites can be shared: Cats, Cats Effect and FS2 (functional programming, async runtime, streaming); Http4s, Tapir and Smithy4s (HTTP and shared API definitions); Circe and uPickle (JSON); MUnit and Weaver (testing, identical on every target).

Check your understanding