Java 24: Structured Concurrency

With Java 24, Structured Concurrency moves closer to becoming a first-class feature in the Java platform. This is currently a preview language feature.

Traditional concurrency in Java often results in fragmented and error-prone code, where related threads are launched independently and can be hard to manage or coordinate. For example, to fetch a user and order in parallel, and then process the results, you would typically use an ExecutorService as shown below:

ExecutorService executor = Executors.newFixedThreadPool(2);
Future<String> userFuture = executor.submit(() -> fetchUser());
Future<String> orderFuture = executor.submit(() -> fetchOrder());
String user = userFuture.get();   // blocks until user is fetched
String order = orderFuture.get(); // blocks until order is fetched
String result = process(user, order);

The downsides of the above approach are:

• If one task fails, the other continues unless manually cancelled
• The executor and tasks outlive the method unless explicitly shut down
• You must manage the executor, handle exceptions, and ensure cleanup

Structured Concurrency abstracts much of this complexity, allowing you to focus on what your code is doing rather than how to coordinate threads. It enforces a hierarchical structure, in which tasks spawned together must complete together, much like local variables within a method.

StructuredTaskScope
Here is an example of using the StructuredTaskScope API:

try (var scope = new StructuredTaskScope<String>()) {
  Subtask<String> userTask = scope.fork(() -> fetchUser());
  Subtask<String> orderTask = scope.fork(() -> fetchOrder());

  scope.join(); // Wait for all subtasks to complete

  String user = userTask.get();
  String order = orderTask.get();

  System.out.println("user: " + user);
  System.out.println("order: " + order);
}

StructuredTaskScope has two subclasses, ShutdownOnSuccess and ShutdownOnFailure, to control how the scope reacts to task completion or failure.

StructuredTaskScope.ShutdownOnFailure
With this policy, if any task fails, the scope cancels the remaining tasks, and propagates the exception when throwIfFailed() is called.

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
  Subtask<String> userTask = scope.fork(() -> fetchUser());
  Subtask<String> orderTask = scope.fork(() -> fetchOrder());

  // wait for all subtasks to complete, or one to fail
  scope.join();
  
  // throw if any subtask failed 
  scope.throwIfFailed();

  String user = userTask.get();
  String order = orderTask.get();

  System.out.println("user: " + user);
  System.out.println("order: " + order);
}

StructuredTaskScope.ShutdownOnSuccess
This policy is the opposite — it stops once one task succeeds, cancelling the others. It's great when you want the first successful result and don't care about the rest.

try (var scope = new StructuredTaskScope.ShutdownOnSuccess<String>()) {
  scope.fork(() -> fetchFromPrimary());
  scope.fork(() -> fetchFromBackup());

  // wait for any subtask to complete, or all to fail
  scope.join();

  // get the result of the first task that completed successfully,
  // or throw an exception if none did
  System.out.println(scope.result()); 
}

Java 24: Scoped Values

Java 24 introduces Scoped Values, a powerful alternative to ThreadLocal that offers better performance and cleaner code for managing per-thread data. This is a preview language feature.

Here's an example of ScopedValue in action:

private static final ScopedValue<String> USER_ID = ScopedValue.newInstance();

public void handle(Request req, String userId) {
  ScopedValue.where(USER_ID, userId)
    .run(() -> handle(req));
}

private void handle(Request req) {
  String data = getData(req);
  // Do something else
}

private String getData(Request req) {
  return runQuery(req, USER_ID.get());
}

As shown above, ScopedValue provides a means to pass data (the userId) securely to a faraway method without using method parameters. The faraway method can access the data via the ScopedValue object. This eliminates the need to pass additional parameters explicitly through multiple method calls.

ScopedValue vs ThreadLocal

• Scoped Values are immutable once set inside ScopedValue.where(...). On the other hand, ThreadLocal allows values to be changed at any time, which can lead to inconsistent state across different parts of a request.
• Scoped Values are automatically removed after the scope ends, whereas ThreadLocal requires an explicit call to remove() to avoid memory leaks, especially in thread pools.
• Scoped Values bind data to a specific execution scope, ensuring that when a new task starts on a thread, it doesn’t inherit values from a previous request. ThreadLocal stores data at the thread level, meaning values persist across multiple tasks when using a thread pool.
• Scoped Values work well with virtual threads and structured concurrency APIs.

For comparison, here's the same example using ThreadLocal:

private static final ThreadLocal<String> USER_ID = new ThreadLocal<>();

public void handle(Request req, String userId) {
  try {
    USER_ID.set(userId);
    handle(req);
  } finally {
    USER_ID.remove(); // to prevent memory leaks
  }
}

private void handle(Request req) {
  String data = getData(req);
  // Do something else
}

private String getData(Request req) {
  return runQuery(req, USER_ID.get());
}

While ThreadLocal still has its uses, most new applications will benefit from Scoped Values’ immutability, automatic cleanup, and better thread management.

Java 24: Primitive Types in Patterns, instanceof, and switch

Java 24 introduces enhancements to pattern matching by allowing primitive types in all pattern contexts, and extending instanceof and switch to work with all primitive types. This is a preview language feature.

Previously, pattern matching for switch only supported reference types such as Integer i, but now it supports primitive types too. For example:

int i = 100;
String s = switch(i) {
  case 1 -> "one";
  case 2 -> "two";
  case int i when i > 2 -> "too big";
  default -> "unsupported";
}

Similarly, instanceof has been enhanced to support primitives, as shown in the example below:

int i = 1;
if (i instanceof byte b) {
  // i has been cast to byte and assigned to b
}

Related posts:
Java 19: Record Patterns
Java 17: Pattern Matching for Switch
Java 14: Pattern Matching for instanceof

JavaScript Blobs

A Blob (Binary Large Object) is a data structure used to store raw data. It can be created using the Blob constructor. For instance:

const myBlob = new Blob(['Hello, world!'], { type: 'text/plain' });

You can use Blobs to create URLs, which can be directly embedded into HTML documents. For example, you can create a Blob containing text data and then generate a download link for it. When the user clicks this link, they can download the Blob content as a file. This is shown below:

<html>
<head/>
<body>
  <h1>Download Blob Example</h1>

  <script>
    const createDownloadLink = (content, filename) => {
      // Create a Blob from the content
      const blob = new Blob([content], { type: 'text/plain' });
      
      // Create a URL for the Blob
      const url = URL.createObjectURL(blob);
      
      // Create an <a> element
      const a = document.createElement('a');
      a.href = url;
      a.download = filename;
      a.textContent = `Download ${filename}`;

      // append to body
      document.body.appendChild(a);
      
      // revoke URL after some time or on user action
      // URL.revokeObjectURL(url); 
    }
    createDownloadLink('some content', 'example.txt');
  </script>
</body>
</html>

You can also use a Blob to dynamically generate code and create JavaScript files on-the-fly! Here’s an example of how to create a Web Worker from a Blob:

<html>
<head/>
<body>
  <h1>Web Worker Blob Example</h1>
  <p id="result"></p>

  <script>
    const workerScript = `
      onmessage = e => {
        postMessage(e.data * 2);
      };
    `;

    const blob = new Blob([workerScript], { type: 'application/javascript' });
    const url = URL.createObjectURL(blob);
    const worker = new Worker(url);

    worker.onmessage = e => {
      document.getElementById('result').textContent = 'Worker result: ' + e.data;
      URL.revokeObjectURL(url); // Clean up Blob URL
    };

    worker.postMessage('2');
  </script>
</body>
</html>

Shared Web Workers

In my previous post, I discussed how Web Workers can be used to enhance the responsiveness of web applications by offloading resource-intensive computations to run in the background, thus preventing them from blocking the main thread. Today, let's look into Shared Web Workers and how they can further boost your app's efficiency.

Shared Web Workers are a special type of Web Worker that can be accessed from multiple browsing contexts, such as different tabs, windows, or iframes. This shared access can facilitate various functionalities, including real-time communication, managing shared state, and caching data across multiple tabs or windows. By reusing a single worker instance, Shared Web Workers help reduce memory consumption and improve performance compared to creating a new worker for each context.

The following example shows the basics of using a Shared Web Worker.

1. Create the Shared Web Worker Script

Shared Web Workers use ports to communicate. You need to handle the onconnect event to establish communication with the port and the onmessage event to process incoming messages.

// worker.js
onconnect = (event) => {
  const port = event.ports[0];
  port.onmessage = (e) => {
    port.postMessage(e.data[0] + e.data[1]);
  };
};

2. Use the Shared Web Worker in Your Main Script

In your main script, initialise the Shared Web Worker. This can be done from multiple scripts or HTML pages. Once created, any script running on the same origin can access the worker and communicate with it. The various scripts will use the same worker for tasks, even if they are running in different windows.

<html>
  <head/>
  <body>
    <h1>Shared Web Worker Example</h1>
    <p id="result">Computing...</p>
    <script>
      const worker = new SharedWorker('path/to/worker.js');
      worker.port.onmessage = (e) => {
        document.getElementById('result').textContent = `Result: ${e.data}`;
      };
      worker.port.postMessage([1, 2]);
    </script>
  </body>
</html>

Communication between the main script and the Shared Web Worker is done using the port.postMessage method to send messages and the port.onmessage event to receive messages.

Related posts:
Web Workers

Web Workers

Web Workers allow you to perform resource-intensive computations in background threads, without blocking the main thread that handles user interactions and UI updates. This makes it possible to perform tasks such as data processing, complex calculations, and large data fetching asynchronously, keeping your web application responsive.

The following example shows the basics of using a web worker to perform a simple "sum" calculation.

1. Create a Web Worker Script

First, create your web worker script:

// worker.js
onmessage = (e) => {
  const workerResult = e.data[0] + e.data[1];
  postMessage(workerResult);
};

As shown above, the web worker performs the computation in the onmessage event handler and then calls postMessage, to post the result back to the main thread.

2. Invoke the Web Worker in Your Main Script

In the main script, initialise the web worker and invoke it with some data:

<html>
  <head/>
  <body>
    <h1>Web Worker Example</h1>
    <p id="result">Computing...</p>
    <script>
      const worker = new Worker('worker.js');
      worker.onmessage = (e) => {
        document.getElementById('result').textContent = `Result: ${e.data}`;
      };
      worker.postMessage([1, 2]);
    </script>
  </body>
</html>

Communication between the main script and the web worker is done using the postMessage method to send messages and the onmessage event to receive messages.

React Example

Here is how you would do it in React:

import React, { useState } from 'react';

const App = () => {
  const [result, setResult] = useState(null);

  const handleClick = () => {
    // Create a new Web Worker
    const worker = new Worker(new URL('./worker.js', import.meta.url));

    // Set up message handler
    worker.onmessage = (e) => {
      setResult(e.data);
      worker.terminate(); // Clean up the worker
    };

    // send data to the worker
    worker.postMessage([1, 2]);
  };

  return (
    <div>
      <h1>Simple Web Worker Example</h1>
      <button onClick={handleClick}>Start Computation</button>
      {result !== null && <p>Result from Worker: {result}</p>}
    </div>
  );
};
export default App;

Calling Python Functions from kdb+/q with PyKX

PyKX allows you to call Python functions from kdb+/q (and vice versa), enabling powerful data analysis using the rich ecosystem of libraries available in Python. In this post, I will show how you can invoke a Lasso Regression function in Python by passing a table from a q script.

1. Install PyKX

pip install pykx

2. Create a Python (.p) file

Create a Python file called lasso.p containing a function that takes a Pandas DataFrame, performs Lasso regression (using the scikit-learn machine learning library), and returns a vector of coefficients.

# lasso.p

import numpy as np
import pandas as pd
from sklearn.linear_model import Lasso

def lasso_regression(df):
    X = df.iloc[:, :-1]
    y = df.iloc[:, -1]

    # Perform Lasso regression
    lasso = Lasso(alpha=0.1)
    lasso.fit(X, y)

    coefficients = np.append(lasso.coef_, lasso.intercept_) 
    return coefficients

3. Invoke the Python function from a q script

Next, write a q script that generates a table of random data and invokes the Python function with it.

// Load pykx and the python file
\l /path/to/python/site-packages/pykx/pykx.q
\l lasso.p

// Create a sample table with random x and y values
n:100;
x:n?10f;
y:2*x+n?2f;
data:([]x;y);

// Call the python function
qfunc:.pykx.get[`lasso_regression;<];
coefficients:qfunc data;

Conversion of data types between kdb+/q and Python

When transferring data between q and Python, PyKX applies "default" type conversions. For instance, tables in q are automatically converted to Pandas DataFrames, and lists are converted to NumPy arrays. You can call .pykx.setdefault to change the default conversion type to Pandas, Numpy, Python, or PyArrow. PyKX also provides functions to convert q data types to specific Python types, such as .pykx.tonp which tags a q object to be converted to a NumPy object. The following code illustrates type conversion:

q) .pykx.util.defaultConv
"default"

// lists are converted to NumPy arrays by default
q) .pykx.print .pykx.eval["lambda x: type(x)"] til 10
<class 'numpy.ndarray'>

// tables are converted to Pandas DataFrames by default
q) .pykx.print .pykx.eval["lambda x: type(x)"] ([] foo:1 2)
<class 'pandas.core.frame.DataFrame'>

// change default conversion to NumPy
q) .pykx.setdefault["Numpy"]

// tables are NumPy arrays now
q) .pykx.print .pykx.eval["lambda x: type(x)"] ([] foo:1 2)
<class 'numpy.recarray'>

// change default conversion to Python
q) .pykx.setdefault["Python"]

// tables are converted to dict when using Python conversion
q) .pykx.print .pykx.eval["lambda x: type(x)"] ([] foo:1 2)
<class 'dict'>

// tag a q object as a Pandas DataFrame
q) .pykx.print .pykx.eval["lambda x: type(x)"] .pykx.topd ([] foo:1 2)
<class 'pandas.core.frame.DataFrame'>

Java 22: Stream Gatherers

Java 22 introduces Stream Gatherers, a preview language feature, that allows you to build complex stream pipelines using custom intermediate operations, such as grouping elements based on specific criteria, selecting elements with intricate conditions, or performing sophisticated transformations. Furthermore, Stream Gatherers offer seamless integration with parallel stream processing, ensuring optimal performance even in parallel execution scenarios.

There are built-in gatherers like fold, mapConcurrent, scan, windowFixed, and windowSliding, but you can also define your own custom gatherers.

Here is an example using the windowFixed gatherer to group elements in a stream into sliding windows of a specified size:

IntStream.range(0,10)
  .boxed()
  .gather(Gatherers.windowFixed(2))
  .forEach(System.out::println);

// Result:
[0, 1]
[2, 3]
[4, 5]
[6, 7]
[8, 9]

Java 22: Statements Before super(...)

Java 22 brings forth a new preview language feature: the ability to include statements before the super() call in constructors.

Traditionally, Java constructors have had a strict rule: the super() call, which invokes the superclass constructor, must always be the first statement in a subclass constructor. This rule, while ensuring proper initialisation order, sometimes led to verbose or convoluted constructor implementations, especially when additional setup was required before invoking the superclass constructor.

With the introduction of JDK 22, this limitation has been relaxed with the introduction of pre-super statements, which allow you to validate and prepare arguments before the super() call. This also facilitates fail-fast scenarios, because you can perform rigorous argument validation or exception handling before superclass instantiation.

Here is an example:

class Shape {
  private final String color;

  Shape(String color) {
    this.color = color;
  }
}

class Rectangle extends Shape {
  private final double length;
  private final double width;

  Rectangle(String color, double length, double width) {
    if (length <= 0 || width <= 0) {
      throw new IllegalArgumentException("Dimensions must be positive");
    }
    super(color);
    this.length = length;
    this.width = width;
  }
}

In this example, before invoking the superclass constructor, a pre-super statement validates the dimensions of the rectangle, ensuring they are positive.

Java 22: Unnamed Variables and Patterns

Java 22 introduces Unnamed Variables & Patterns. Unnamed variables are placeholders denoted by the underscore character (_) that stand in for variable names, particularly in situations where the variable's identifier is insignificant or redundant. They can be declared in several contexts, including local variable declarations, catch clauses, lambda expressions, and more. By omitting explicit variable names in scenarios where the variable name serves no functional purpose, code becomes more succinct, reducing unnecessary verbosity and aiding readability.

Here are a few examples of unnamed variables in action:

For-loop:

for (Order _ : orders) {
  doSomething();
}

Assignment statement:

Queue<Integer> q = ... // x1, y1, z1, x2, y2, z2, ...
var x = q.remove();
var y = q.remove();
var _ = q.remove();

Lambda expressions:

list.stream().mapToInt(_ -> 1).sum();

Exception handling:

String s = ...
try {
  int i = Integer.parseInt(s);
} catch (NumberFormatException _) {
  System.out.println("Invalid number: " + s);
}

Try-with-resources:

try (BufferedReader _ = new BufferedReader(...)) {
  System.out.println("File opened successfully.");
} catch (IOException _) {
  System.err.println("An error occurred while opening the file.");
}

Unnamed pattern variables:

switch (shape) {
  case Circle _ -> process(shape, 0);
  case Triangle _ -> process(shape, 3);
  case Rectangle _ -> process(shape, 4);
  case var _ -> System.out.println("Unknown shape");
}

Unnamed Patterns

Unnamed Patterns provide an elegant solution when you need to match a pattern without extracting specific components. If you have nested data structures, such as records within records, with unnamed patterns, you can focus on extracting the necessary components without cluttering your code with unnecessary variable assignments.

record Address(String city, String country) {}
record Person(String name, int age, Address address) {}

if (person instanceof Person(var name, _, Address(var city, _))) {
  System.out.println(name + " lives in " + city);
}

So, the next time you encounter a situation where the variable name seems inconsequential, consider using an unnamed variable to streamline your code.

Python: Running Tasks in Parallel

The concurrent.futures module can be used to run tasks in parallel in Python. Here is an example:

import concurrent.futures
from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor(max_workers = 10) as executor:
    futures = [executor.submit(perform_task, task) for task in tasks]
	
results = [future.result() for future in futures]

The ProcessPoolExecutor class uses a pool of processes to execute tasks asynchronously. The submit function immediately returns a Future object, and you can call future.result(), which will block until the task has completed.

Note that there is also a ThreadPoolExecutor class which uses a pool of threads; however, due to the Global Interpreter Lock, only one thread can execute python bytecode at any one time, which means that you will not achieve any parallelisation in most cases.

Java 21: Sequenced Collections

Introduced in Java 21, a SequencedCollection is a Collection whose elements have a defined encounter order i.e. it has first and last elements, and the elements between them have successors and predecessors. Some examples include List, Deque, SortedSet, and LinkedHashSet.

The SequencedCollection interface provides methods to add, retrieve, and remove elements at either end of the collection. It also has a reversed() method which provides a reverse-ordered view of the original collection.

Similarly, the new SequencedMap interface is a map that has a well-defined encounter order, supports operations at both ends, and is reversible. Examples include LinkedHashMap and TreeMap.

Example usage:

var set = new LinkedHashSet<String>(Arrays.asList("a", "b", "c"));

set instanceof SequencedCollection
==> true

set.getFirst()
==> "a"

set.getLast()
==> "c"

set.reversed()
==> [c, b, a]

Java 21: Unnamed Classes

Java 21 introduces Unnamed Classes (a preview language feature) that allow you to write small programs without having an enclosing class declaration.

Here is the classic Hello World program that we were all taught when starting to learn Java:

public class HelloWorld { 
  public static void main(String[] args) { 
    System.out.println("Hello, World!");
  }
}

There is a lot of clutter here. Using an unnamed class, this can be simplified to:

void main() { 
  System.out.println("Hello, World!");
}

Not only is the enclosing class not required, but the main method has also been enhanced so that it does not need to be public, static or require any arguments.

You can also add fields and methods to an unnamed class, as shown below:

private static final String GREETING = "Hello, World!";

private String getGreeting() {
  return GREETING;
}

void main() { 
  System.out.println(getGreeting());
}

Since an unnamed class cannot be instantiated or referenced by name, it is only useful as a standalone program or as an entry point to a program.

Java 21: String Templates

In Java 21, String Templates have been introduced as a preview language feature, that allow text and expressions to be composed safely and efficiently, without using the + operator.

Here is an example:

int x = 5, y = 6;

String s = STR."\{x} plus \{y} is equal to \{x + y}";

// evaluates to: "5 plus 6 is equal to 11"

In this example, STR is a template processor. The template is \{x} plus \{y} is equal to \{x + y} and \{x} is one of the embedded expressions in the template. The STR template processor is defined in the Java Platform (and is automatically imported into every Java source file), and it performs string interpolation by evaluating the embedded expressions.

More examples:

// you can invoke methods, access fields, use ternaries
String s = STR."\{user.name}: Access \{user.hasAccess() ? "Granted" : "Denied"}";

// multi-line template expression
String xml = STR."""
<book>
  <author>\{author}</author>
  <title>\{title}</title>
</book>
""";

FMT Template Processor

FMT is like STR but it also interprets format specifiers which appear to the left of embedded expressions. For example:

double val = 4999.4567;

FormatProcessor.FMT."The value is %,.2f\{val}";

// evaluates to: "The value is 4,999.46"

It's quite easy to create your own Template Processor by implementing StringTemplate.Processor. This is useful if you want to validate inputs before composing the string. It's also possible to return an object of any type, not just String. For instance, a SQL Template processor could first sanitise the input to prevent a SQL injection attack, and then return a PreparedStatement instead of a String.

Linear and Polynomial Regression in kdb+/q

In this post, I'll describe how you can implement linear and polynomial regression in kdb+/q to determine the equation of a line of best fit (also known as a trendline) through the data on a scatter plot.

Consider the following scatter plot:

Our aim is to estimate a function of a line that most closely fits the data.

The vector of estimated polynomial regression coefficients (using ordinary least squares estimation) can be obtained using the following formula (for information about how this formula is derived, see Regression analysis [Wikipedia]):

b = (XTX)−1XTy

This can be translated into q as follows:

computeRegressionCoefficients:{
    xt:flip x;
    xt_x:xt mmu x;
    xt_x_inv:inv xt_x;
    xt_y:xt mmu y;
    xt_x_inv mmu xt_y}

Linear:
In order to perform linear regression, we have to first create a matrix X with a column of 1s and a column containing the x-values. The output of linear regression will be a vector of 2 coefficients and the equation of the trendline will be of the form: y = b1x + b0

computeLinearRegressionCoefficients:{
    computeRegressionCoefficients[flip (1f;x);y]}

Polynomial:
In order to fit a polynomial line, all we have to do is take the matrix X from the linear regression model and add more columns corresponding to the order of the polynomial desired. For example, for quadratic regression, we will add a column for x² on the right side of the matrix X. The output of quadratic regression will be a vector of 3 coefficients and the equation of the curve will be of the form: y = b2x2 + b1x + b0

// quadratic
computeQuadraticRegressionCoefficients:{
    computeRegressionCoefficients[flip (1f;x;x*x);y]}

// cubic
computeCubicRegressionCoefficients:{
    computeRegressionCoefficients[flip (1f;x;x*x;x*x*x);y]}

// generalisation of polynomial regression for any order
computePolynomialRegressionCoefficients:{[x;y;order]
    computeRegressionCoefficients[flip x xexp/: til order+1;y]}

Related post:
Matrix Operations in kdb+/q

Matrix Operations in kdb+/q

In q, a matrix (an array of m x n numbers) is represented as a list of lists. For example, here is a matrix with 2 rows and 3 columns:

q)A:(1 2 3;4 5 6)
q)A
1 2 3
4 5 6

Matrix Addition and Subtraction
If A and B are matrices of the same size, then they can be added and subtracted. To find the entries of A + B, you simply add the corresponding entries of A and B. To find A - B, subtract corresponding entries. If A and B have different sizes, you will get a 'length error.

q)A:(1 2;3 4)
q)B:(5 6;7 8)
q)A+B
6  8
10 12
q)B-A
4 4
4 4
q)C:(1 1 1;2 2 2)
q)A+C
'length
  [0]  A+C
        ^

Scalar Multiplication
If A is a matrix and k is a scalar, then the matrix kA is obtained by multiplying each entry of A by k.

q)A:(1 2;3 4)
q)k:2
q)k*A
2 4
6 8

Matrix Multiplication
First, in order to multiply two matrices A and B, the number of columns of A must match the number of rows of B. If A is m x n and B is n x p, then the size of the product matrix AB will be m x p. In order to calculate AB, you have to take the dot product (multiply corresponding numbers and then add them up) of each row vector in A and the corresponding column vector in B. This can be done in q using the mmu (or $) operator.

q)A:(1 2f;3 4f)
q)B:(5 6f;7 8f)
q)A
1 2
3 4
q)B
5 6
7 8
q)A mmu B
19 22
43 50

Identity Matrix
This is a square matrix with 1s on the diagonal and 0s everywhere else.

q)I:{`float${x=/:x}til x}
q)I 3
1 0 0
0 1 0
0 0 1

Matrix Inverse
The inverse of A is A^-1 if AA^-1=A^-1A=I, where I is the identity matrix. Use the inv function to find the inverse of a matrix.

q)A:(1 2f;3 4f)
q)inv A
-2  1
1.5 -0.5
q)A mmu inv A
1 1.110223e-016
0 1

Matrix Tranpose
The tranpose A^T of a matrix A is a flipped version of the original matrix which is obtained by changing its rows into columns (or equivalently, its columns into rows). This can be done by using the flip operation in q.

q)A:(1 2 3;4 5 6)
q)flip A
1 4
2 5
3 6

Upper Triangular Matrix
This is a square matrix in which all the entries below the main diagonal are zero.

q)upperTriangle:{`float${x<=\:x}til x}
q)upperTriangle 3
1 1 1
0 1 1
0 0 1

Lower Triangular Matrix
This is a square matrix in which all the entries above the main diagonal are zero.

q)lowerTriangle:{`float${x>=\:x}til x}
q)lowerTriangle 3
1 0 0
1 1 0
1 1 1

kdb+/q - Converting a CSV String into a Table

I often find myself wanting to create quick in-memory tables in q for testing purposes. The usual way to create a table is by specifying lists of column names and values, or flipping a dictionary, as shown below:

([] name:`Alice`Bob`Charles;age:20 30 40;city:`London`Paris`Athens)

// or, using a dictionary:

flip `name`age`city!(`Alice`Bob`Charles;20 30 40;`London`Paris`Athens)

As you can see, it's quite difficult to visualise the table being created using this approach. That's why I sometimes prefer to create a table from a multi-line CSV string instead, using the 0: operator, as shown below:

("SIS";enlist",") 0:
"name,age,city
Alice,20,London
Bob,30,Paris
Charles,40,Athens
"

name    age city
------------------
Alice   20  London
Bob     30  Paris
Charles 40  Athens

Note that you can also load from a CSV file:

("SIS";enlist",") 0: `$"/path/to/file.csv"

Related post:
kdb+/q - Reading and Writing a CSV File

Java 20: Record Patterns in For Loops

Previously, I wrote about "Record Patterns" introduced in Java 19, that allow you to "deconstruct" records and access their components directly.

In Java 20 (released a couple of weeks ago!), record patterns have been enhanced so that they can also be used in for loops.

Here is an example that uses a nested record pattern in a for loop to print out a list of records:

record Author(String firstName, String lastName) {}
record Book(String title, Author author, double price) {}

static void printBooks(List<Book> books) {
  for (Book(var title, Author(var firstName, var lastName), var price): books) {
    System.out.printf("%s by %s %s for %.2f\n", title, firstName, lastName, price);
  }
}

Related post:
Java 19: Record Patterns

Java: Collecting a Stream into an Existing Collection

The following snippet shows how you can collect a stream into an existing collection using Collectors.toCollection:

stream.collect(Collectors.toCollection(() -> existingCollection));

You may also be interested in reading my previous post about collecting a stream into an unmodifiable collection.

Java 19: Virtual Threads

Java 19 introduces Virtual Threads, which are lightweight threads designed to improve application throughput. This is a preview language feature so must be enabled using --enable-preview.

As you know, the JDK implements platform threads (java.lang.Thread) as thin wrappers around operating system (OS) threads. OS threads are expensive to create, and the number of threads is limited to the number of OS threads that the underlying hardware can support. A platform thread captures the OS thread for the code's entire lifetime.

On the other hand, a virtual thread is an instance of java.lang.Thread that is not tied to a particular OS thread and does not capture the OS thread for the code's entire lifetime. A virtual thread consumes an OS thread only while it performs calculations on the CPU. This means that several virtual threads can run their Java code on the same OS thread, effectively sharing it. When code running in a virtual thread calls a blocking I/O operation, the JVM performs a non-blocking OS call and automatically suspends the virtual thread until it can be resumed later.

Virtual threads help to improve the throughput of thread-per-request style server applications in particular because such applications consist of a great number of concurrent tasks that are not CPU bound and spend much of their time waiting.

Here is an example which creates 10,000 virtual threads; however, the JDK runs the code on perhaps only one OS thread. If we were using 10,000 platform threads (and thus 10,000 OS threads) instead, the program might crash, depending on the hardware available. Virtual threads are cheap and plentiful, and there is no need to pool them.

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
  IntStream.range(0, 10_000).forEach(i -> {
    executor.submit(() -> {
      Thread.sleep(Duration.ofSeconds(1));
      return i;
    });
  });
}

The java.lang.Thread class has been updated with new methods to create virtual and platform threads, such as:

// create a new unstarted virtual thread named "foo".
Thread.ofVirtual().name("foo").unstarted(runnable);

// create and start a virtual thread
Thread.startVirtualThread(runnable)