Franz: macOS Client for Apache Kafka

8.8.0.8

Franz: macOS Client for Apache Kafka

Franz is a native macOS client for Apache Kafka. It helps you manage your Kafka clusters, topics and consumer groups and it provides convenient functionality for monitoring the data being published to topics.

1 Connections

When you start Franz, you are presented with the Welcome Window. From the Welcome Window you can connect to servers you’ve previously used or create new connections.

You can access the Welcome Window using the “Window” -> “Welcome to Franz” menu item or by pressing ⇧ ⌘ 1.

1.1 Security

Connection metadata is stored inside Franz’ internal metadata database, but passwords are stored in the macOS Keychain.

1.2 Workspaces

When you connect to a Kafka cluster, a Workspace Window is opened for that cluster. All operations within the workspace operate on the same connection. When you close a Workspace Window, all of its associated connections and interface objects are closed.

1.3 Topics

From the Workspace Window sidebar, you can select topics to view general information about them, to browse through their record data and publish new data, or to view their configuration.

You can publish new data on a topic by pressing the plus icon on the top right corner of the Workspace Window toolbar.

1.3.1 Information Tab

The Information tab (⌘ 1) displays general information about the selected topic.

1.3.2 Records Table Tab

The Records Table tab (⌘ 2) on a topic lets you stream live data being published on a topic or jump to any offset you like and paginate through the data manually.

When you open the Records Table tab on a topic, it immediately starts streaming recent data into the table. You can stop this by pressing the “Toggle Live Mode” button on the bottom left corner of the table. You can configure how much data is requested from the topic on each fetch by click the “Options...” button in the bottom right, and you can manually load more data by pressing the “Load More Records...” button.

From the “Options...” popover, you can also jump to any offset you like. See Jump Popover for details.

You can right-click on any record with a non-null key to publish a tombstone for it. Additionally, you can drag and drop any non-null key or value from the table to any application that accepts files to export the dragged value. You can use the “Key Format” and “Value Format” options from the “Options...” popover to control what format the columns are exported as.

Double-clicking any record will bring up its Record Detail Window.

1.3.3 Records Table Scripting

You can control the values displayed in the Records Table by writing Lua scripts. With the Records Table for a topic selected, press the scripting button – located in the center bottom of the table – to bring up the scripting window. Using the scripting window, you can edit the transform function to control how data is presented in the Records Table.

To activate and deactivate a script, press the bolt icon in the scripting window toolbar or use the ⌘ ↩ keyboard shortcut. After a script is activated, any changes made to the text of the script will cause it to be deactivated.

The record argument to the transform function is a Lua table with the following fields:

field	description
partition_id	the partition the record was published to
offset	the record’s offset as a non-negative integer
timestamp	the record’s timestamp in milliseconds since the UNIX epoch
key	the record’s key as a string or nil
value	the record’s value as a string or nil

You may modify any of these fields to control how the record is displayed in the Records Table. Changing the data types of these fields is prohibited and will lead to an error when data gets loaded.

Returning nil from the transform function will cause the record to be skipped in the Records Table. You can leverage this to, for example, filter records by partition:

function script.transform(record)
  if record.partition_id ~= 2 then
    return nil
  end
  return record
end

Within the scripting environment, a json table is provided with functions for encoding and decoding JSON data. For example, the following script can be used to read the example property of the record’s JSON value:

local script = {}

function script.transform(record)
record.value = json.decode(record.value).example
return record
end

return script

See the Scripting Reference for a list of all the functionality available within the scripting environment.

1.3.4 Jumping to Offsets

From the “Options...” popover of a Records Table, push the “Jump...” button to get to the Jump Popover (⇧ ⌘ J). From there, you can reset the record iterator to various offsets, as described below.

Earliest

Queries each partition for its earliest offset and moves the iterator back. This is slightly different than explicitly resetting all partitions to offset 0 as the first offset on a partition might not necessarily be 0 (as in the case of compacted records). Functionally, however, it has the same effect: the iterator will start iterating through records from the very beginning of the topic’s history.

Timestamp

Queries each partition for the first offset on or after the given date and time and moves the iterator there. For partitions where the timestamp represents a time after the latest offset, it makes an additional query to find the latest offset.

Recent

Queries each partition for its latest offset and moves the iterator to that position, minus the requested delta.

Latest

Queries each partition for its latest offset and moves the iterator forward.

Offset

Moves the iterator to the given offset for every partition. For partitions that are behind the selected offset, no new data will be received until they reach it.

1.3.5 Consumer Groups Tab

The Consumer Groups Tab (⌘ 3) displays the active consumer groups for the selected topic. This is an easy way to discover what groups are actively reading from individual topics.

1.3.6 Configuration Table Tab

The Configuration Table (⌘ 4) tab displays the selected topic’s configuration. Non-default values are presented in bold and sensitive values are hidden by default. You may reveal sensitive values by right clicking on them and pressing the “Reveal” context menu item.

1.4 Record Detail Window

The Record Detail Window displays the contents of individual records. You can configure the default format for the key and the value on a per-topic basis by customizing the “Key Format” and the “Value Format” from the Records Table “Options...” popover.

1.5 Consumer Groups

When you select a consumer group from the Workspace Window sidebar, you are presented with the Consumer Offsets Table. There, you can see member assignments, offsets and lag as well as reset individual offsets by right-clicking any of the entries.

You may only reset offsets if the consumer group is in the empty state.

1.6 Schema Registry

With a Workspace Window in the foreground, you can configure a Schema Registry from the main menu by selecting “Schema Registry” -> “Configure...”. Once a registry is configured, records are automatically converted to JSON according to the schemas found in the registry before being displayed in the Records Table and before being passed to any Lua scripts. To remove a registry, open the configuration window and remove its URL then press “Save”.

See this YouTube video for a live demo.

2 Keyboard Shortcuts

⇧ ⌘ 1 — Displays the Welcome Window.

⇧ ⌘ J — With a topic Records Table visible, turns off live mode (if on) and displays the Jump Popover.

⌘ 1 — With a broker or topic selected, switches to the Information tab.

⌘ 2 — With a topic selected, switches to the Records Table tab. With a broker or consumer group selected, switches to the Configuration tab.

⌘ 3 — With a topic selected, switches to the Consumer Groups Tab.

⌘ 4 — With a topic selected, switches to the Configuration tab.

⌘ R — Within a Workspace Window, reloads the connection metadata.

⌘ T — Within a Workspace Window, duplicates the Workspace in a new tab.

⇧ ⌘ T — Within a Workspace Window, duplicates the Workspace in a new window.

⌘ ↩ — Within a Scripting Window, activates or deactivates the script.

⌘ , — Opens the Preferences Window.

3 Known Issues and Limitations

If any of these limitations are showstoppers for you, please e-mail me at bogdan@defn.io and let me know.

3.1 Compression

The only compression formats currently supported are gzip and lz4.

3.2 Schema Registry

The only type of schema registry currently supported is the Confluent Schema Registry. Protobuf schemas are currently not supported.

4 Scripting Reference

avro.parse(str)	avro.Codec
Decodes the Apache Avro schema represented by str and returns a avro.Codec. Raises an error if the schema is invalid. Use the avro.Codec:read method on the returned codec to decode data.

avro.Codec:read(str)

any

Reads the data in str according to its internal schema. Avro records are represented by Lua tables with keys named after every field. Arrays are represented by integer-indexed tables. Enums are represented by strings. Unions are represented by tables with exactly two keys: a type key referencing the fully-qualified type name of the value and a value key containing the value. Bytes and string values both map to Lua strings. All other primitive values map to Lua values in the way you would expect.
See Decoding Avro Data for an example.

json.decode(str)	table
Decodes the JSON data in str to a Lua table.

json.encode(v)	string
Encodes the Lua value v to JSON.

kafka.parse_committed_offset(record)

tuple

Decodes committed offset data off of the __committed_offsets topic. On failure, returns nil. On success, returns a string representing the type of event that was decoded and a value representing that event. The currently-supported event types are "offset_commit" and "group_metadata". For example:
local script = {}

function script.transform(record)
  local event_type, data = kafka.parse_committed_offset(record)
  if event_type == nil then
    return record
  end
  record.value = tostring(data) -- or json.encode(data)
  return record
end

return script

kafka.record_size(record)	number
Returns the size in bytes of the given record.

math.abs(n)	number
Returns the absolute value of n.

math.acos(n)	number
Returns the arc cosine of n in radians.

math.asin(n)	number
Returns the arc sine of n in radians.

math.atan(y, x)	number
Returns the arc tangent of y/x. The x argument defaults to 1 if not provided.

math.ceil(n)	number
Rounds n towards positive infinity.

math.cos(n)	number
Returns the cosine of n.

math.deg(n)	number
Converts the angle n from radians to degrees.

math.exp(n)	number
Raises the base of natural logarithms to n (e^n).

math.floor(n)	number
Rounds n towards negative infinity.

math.log(n, base)	number
Returns the logarithm of x in the given base. The base argument defaults to e.

math.max(n, ...)	number
Returns the largest number amongst the given set.

math.min(n, ...)	number
Returns the smallest number amongst the given set.

math.rad(n)	number
Converts the angle n from degrees to radians.

math.random(m, n)	number
With no arguments, returns a random number in the range [0.0, 1.0]. With one argument, returns a random number in the range [0.0, m]. With two arguments, returns a random number in the range [m, n].

math.randomseed(x, y)	number
With no arguments, seeds the random number generator arbitrarily. With one argument, seeds the random number generator to x. With two arguments, seeds the random number generator to x ~ y & 0x7FFFFFFF.

math.sin(n)	number
Returns the sine of n.

math.sqrt(n)	number
Returns the square root of n.

math.tan(n)	number
Returns the tangent of n.

math.tointeger(v)	number
Converts v to an integer. Returns false if the value cannot be converted.

msgpack.unpack(str)	any
Decodes the MessagePack-encoded value represented by str to Lua. Arrays and maps are represented as Lua tables. Strings and binary data are represented as Lua strings. Nil is represented as Lua nil. See Decoding MessagePack Data for an example.

string.byte(str, i, j)	table
Returns the bytes in str between i and j. The i argument defaults to 1 and the j argument defaults to the length of str.

string.char(...)	string
Constructs a string from the given bytes.

string.format(fmt, ...)	string
Formats the variable arguments according to fmt. Behaves the same as the standard C function sprintf, except that there is an additional conversion specifier, %q, which quotes literal Lua values.

string.len(str)	number
Returns the length of str.

string.lower(str)	string
Returns a new string with the characters in str lowercased according to the current locale.

string.rep(str, n, sep)	string
Repeats str n times, interspersing sep between repetitions. The sep argument defaults to "".

string.reverse(str)	string
Returns a new string with the characters in str in reverse order.

string.sub(str, i, j)	string
Returns a substring of str starting from i until j. The i argument defaults to 1 and the j argument defaults to the length of str.

string.upper(str)	string
Returns a new string with the characters in str uppercased according to the current locale.

4.1 Examples

4.1.1 Decoding JSON Data

Use json.decode to decode your data.

local script = {}

function script.transform(record)
  local object = json.decode(record.value)
  record.value = tostring(object.field)
  return record
end

return script

4.1.2 Decoding Avro Data

Use avro.parse to convert an Avro Schema into a codec. Then, use that codec to decode your record data.

local script = {}
local schema = [[
  {
    "type": "record",
    "name": "Person",
    "fields": [
      {
        "name": "Name",
        "type": "string"
      },
      {
        "name": "Age",
        "type": "int"
      }
    ]
  }
]]
local person_codec = avro.parse(schema)

function script.transform(record)
  local person = person_codec:read(record.value)
  record.value = person.Name
  return record
end

return script

You can write your schema as a Lua table and convert it to JSON using json.encode. For example, you could rewrite the above example to:

local script = {}
local schema = json.encode(
  {
    type = "record",
    name = "Person",
    fields = {
      { name = "Name", type = "string" },
      { name = "Age",  type = "int" }
    }
  }
)
local person_codec = avro.parse(schema)

function script.transform(record)
  local person = person_codec:read(record.value)
  record.value = person.Name
  return record
end

return script

See this YouTube video for a live demo.

4.1.3 Decoding MessagePack Data

Use msgpack.unpack to decode your data.

local script = {}

function script.transform(record)
  local object = msgpack.unpack(record.value)
  record.value = tostring(object.field)
  return record
end

return script

5 Privacy

Apart from when checking for updates, Franz never phones home for any reason. Automatic Updates can be turned off from the Preferences Window (⌘ ,).

6 Credits

Franz is built using the Racket programming language and distributes its runtime alongside the application. Racket is licensed under the MIT License.

1	Connections
2	Keyboard Shortcuts
3	Known Issues and Limitations
4	Scripting Reference
5	Privacy
6	Credits