Franz: Mac and Windows Client for Apache Kafka🔗

Franz is a native Mac and Windows client for Apache Kafka. It helps you manage your Kafka clusters, topics and consumer groups and it provides convenient functionality for monitoring the data being published to topics.

1 Connections🔗

When you start Franz, you are presented with the Welcome Window. From the Welcome Window you can connect to servers you’ve previously used or create new connections.

On macOS, you can access the Welcome Window using the “Window” -> “Welcome to Franz” menu item or by pressing ⇧ ⌘ 1. On Windows, the window is accessible via the the “Help” -> “Welcome to Franz” menu item or by pressing ⇧ ⌃ 1.

1.1 Security🔗

Connection metadata is stored inside Franz’ internal metadata database, but passwords are stored in the macOS Keychain. On Windows, passwords are encrypted using the Windows CryptProtectData API, meaning passwords can only be decrypted by the user account they were originally encrypted by.

1.2 Workspaces🔗

When you connect to a Kafka cluster, a Workspace Window is opened for that cluster. All operations within the workspace operate on the same connection. When you close a Workspace Window, all of its associated connections and interface objects are closed.

1.3 Topics🔗

From the Workspace Window sidebar, you can select topics to view general information about them, to browse through their record data and publish new data, or to view their configuration.

You can publish new data on a topic by pressing the plus icon on the top right corner of the Workspace Window toolbar.

1.3.1 Information Tab🔗

The Information tab (⌘ 1) displays general information about the selected topic.

1.3.2 Records Table Tab🔗

The Records Table tab (⌘ 2) on a topic lets you stream live data being published on a topic or jump to any offset you like and paginate through the data manually.

When you open the Records Table tab on a topic, it immediately starts streaming recent data into the table. You can stop this by pressing the “Toggle Live Mode” button on the bottom left corner of the table. You can configure how much data is requested from the topic on each fetch by click the “Options...” button in the bottom right, and you can manually load more data by pressing the “Load More Records...” button.

From the “Options...” popover, you can also jump to any offset you like. See Jump Popover for details.

You can right-click on any record with a non-null key to publish a tombstone for it. Additionally, you can drag and drop any non-null key or value from the table to any application that accepts files to export the dragged value. You can use the “Key Format” and “Value Format” options from the “Options...” popover to control what format the columns are exported as.

Double-clicking any record will bring up its Record Detail Window.

1.3.3 Records Table Scripting🔗

You can control the values displayed in the Records Table by writing Lua scripts. With the Records Table for a topic selected, press the scripting button – located in the center bottom of the table – to bring up the scripting window. Using the scripting window, you can edit the transform function to control how data is presented in the Records Table.

To apply a script, press the play icon in the scripting window toolbar or use the ⌘ ⇧ ↩ keyboard shortcut. Applying a script immediately runs the code against the records already loaded in the records table.

To activate and deactivate a script, press the bolt icon in the scripting window toolbar or use the ⌘ ↩ keyboard shortcut. When a script is activated, it is automatically applied to any newly-loaded records in the records table. After a script is activated, any changes made to the text of the script will cause it to be deactivated.

The record argument to the transform function is a Lua table with the following fields:

You may modify any of these fields to control how the record is displayed in the Records Table. Changing the data types of these fields is prohibited and will lead to an error when data gets loaded.

Returning nil from the transform function will cause the record to be skipped in the Records Table. You can leverage this to, for example, filter records by partition:

function script.transform(record)

if record.partition_id ~= 2 then

return nil

end

return record

end

Within the scripting environment, a json table is provided with functions for encoding and decoding JSON data. For example, the following script can be used to read the example property of the record’s JSON value:

local script = {}

function script.transform(record)

record.value = json.decode(record.value).example

return record

end

return script

See the Scripting Reference for a list of all the functions available in the scripting environment.

When applying a script, you can aggregate data in memory by providing a reduce function. For example, the following function counts the number of bytes across all the loaded records’ values:

function script.reduce(record, state)

return (state or 0) + #record.value

end

Once aggregated, you can display the aggregated data by providing a render function. For example, the display the output of the reduce function from above, you can write:

function script.render(state)

return string.format("Total bytes: %d", state)

end

Strings and numbers may be returned directly from the render functions. More complex visualizations are possible using the functions provided by the render object. See Rendering a Bar Chart for an example.

1.3.4 Jumping to Offsets🔗

From the “Options...” popover of a Records Table, push the “Jump...” button to get to the Jump Popover (⇧ ⌘ J). From there, you can reset the record iterator to various offsets, as described below.

Earliest

Queries each partition for its earliest offset and moves the iterator back. This is slightly different than explicitly resetting all partitions to offset 0 as the first offset on a partition might not necessarily be 0 (as in the case of compacted records). Functionally, however, it has the same effect: the iterator will start iterating through records from the very beginning of the topic’s history.

Timestamp

Queries each partition for the first offset on or after the given date and time and moves the iterator there. For partitions where the timestamp represents a time after the latest offset, it makes an additional query to find the latest offset.

Recent

Queries each partition for its latest offset and moves the iterator to that position, minus the requested delta.

Latest

Queries each partition for its latest offset and moves the iterator forward.

Offset

Moves the iterator to the given offset for every partition. For partitions that are behind the selected offset, no new data will be received until they reach it.

1.3.5 Consumer Groups Tab🔗

The Consumer Groups Tab (⌘ 3) displays the active consumer groups for the selected topic. This is an easy way to discover what groups are actively reading from individual topics.

1.3.6 Configuration Table Tab🔗

The Configuration Table (⌘ 4) tab displays the selected topic’s configuration. Non-default values are presented in bold and sensitive values are hidden by default. You may reveal sensitive values by right clicking on them and pressing the “Reveal” context menu item.

Certain configuration options may be editable. The context menu for those entries will have “Edit...” and “Delete” menu items on their context menus. Edits are made in batches and have to be manually applied by pushing the “Apply” button at the bottom of the table. Edits can be reset by pressing the “Reset” button. Switching tabs after making edits will discard all unapplied changes.

1.4 Record Detail Window🔗

The Record Detail Window displays the contents of individual records. You can configure the default format for the key and the value on a per-topic basis by customizing the “Key Format” and the “Value Format” from the Records Table “Options...” popover.

1.5 Consumer Groups🔗

When you select a consumer group from the Workspace Window sidebar, you are presented with the Consumer Offsets Table. There, you can see member assignments, offsets and lag as well as reset individual offsets by right-clicking any of the entries.

You may only reset offsets if the consumer group is in the empty state.

1.6 Schema Registry🔗

With a Workspace Window in the foreground, you can configure a Schema Registry from the main menu by selecting “Schema Registry” -> “Configure...”. Once a registry is configured, records are automatically converted to JSON according to the schemas found in the registry before being displayed in the Records Table and before being passed to any Lua scripts. To remove a registry, open the configuration window and remove its URL then press “Save”.

See this YouTube video for a live demo.

2 Keyboard Shortcuts (macOS)🔗

⇧ ⌘ 1 — Displays the Welcome Window.

⇧ ⌘ J — With a topic Records Table visible, turns off live mode (if on) and displays the Jump Popover.

⌘ 1 — With a broker or topic selected, switches to the Information tab.

⌘ 2 — With a topic selected, switches to the Records Table tab. With a broker or consumer group selected, switches to the Configuration tab.

⌘ 3 — With a topic selected, switches to the Consumer Groups Tab.

⌘ 4 — With a topic selected, switches to the Configuration tab.

⌘ R — Within a Workspace Window, reloads the connection metadata.

⌘ T — Within a Workspace Window, duplicates the Workspace in a new tab.

⇧ ⌘ T — Within a Workspace Window, duplicates the Workspace in a new window.

⌘ ⇧ ↩ — Within a Scripting Window, runs the script against the current batch of loaded records.

⌘ ↩ — Within a Scripting Window, activates or deactivates the script.

⌘ , — Opens the Preferences Window.

3 Keyboard Shortcuts (Linux & Windows)🔗

⇧ ⌃ 1 — Displays the Welcome Window.

⇧ ⌃ N — Within a Workspace Window, duplicates the Workspace in a new window.

⌃ R — Within a Workspace Window, reloads the connection metadata.

⌃ ⇧ J — Within a Scripting Window, opens the Jump to Line dialog.

⌃ ⇧ ↩ — Within a Scripting Window, runs the script against the current batch of loaded records.

⌃ ↩ — Within a Scripting Window, activates or deactivates the script.

⌃ ; — Within a Workspace Window, opens the Preferences Window.

4 Known Issues and Limitations🔗

If any of these limitations are showstoppers for you, please e-mail me at bogdan@defn.io and let me know.

4.1 Schema Registry🔗

The only type of schema registry currently supported is the Confluent Schema Registry.

5 Scripting Reference🔗

avro.parse(str)

avro.Codec

Decodes the Apache Avro schema represented by str and returns a avro.Codec. Raises an error if the schema is invalid.

Use the avro.Codec:read method on the returned codec to decode data.

avro.Codec:read(str)

any

Reads the data in str according to its internal schema. Avro records are represented by Lua tables with keys named after every field. Arrays are represented by integer-indexed tables. Enums are represented by strings. Unions are represented by tables with exactly two keys: a type key referencing the fully-qualified type name of the value and a value key containing the value. Bytes and string values both map to Lua strings. All other primitive values map to Lua values in the way you would expect.

See Decoding Avro Data for an example.

file:close()

void

Closes the file.

file:isclosed()

bool

Returns true when the file is closed and false otherwise.

file:flush()

void

Flushes any pending output to the file.

file:lines(...)

function

Returns a function that iterates over lines in the file according to the given read instructions. If no read instructions are given, defaults to "l".

local f = io.input("/etc/passwd")

for l in f:lines() do

print(l)

end

See file:read for a description of the supported read instructions.

file.path

string

The file’s path on disk.

file:read(...)

...

Reads bytes from the file according to the given set of read instructions. Returns one string for every instruction given.

Instruction	Effect
"a"	Reads the entire file.
"*all"	Same as "a".
"l"	Reads one line from the file.
number	Reads number number of bytes from the file.

file:write(...)

void

Writes the given set of values to the file. Non-string values are converted to strings using tostring before writing.

io.close(file)

void

Closes the given file.

io.input(handle_or_path)

file

Opens handle_or_path for reading.

io.open(path, mode)

file

Opens the file at path in the given mode. Supported modes are "r" for reading and "w" for writing.

io.output(handle_or_path)

file

Opens handle_or_path for writing.

io.tmpfile()

file

Returns a temporary file, ready for writing.

io.type(f)

string

Returns "closed file" when f is closed, or "file" when it is open.

io.write(...)

void

Writes ... to standard output.

json.decode(str)

table

Decodes the JSON data in str to a Lua table.

json.encode(v)

string

Encodes the Lua value v to JSON.

kafka.parse_committed_offset(record)

tuple

Decodes committed offset data off of the __committed_offsets topic. On failure, returns nil. On success, returns a string representing the type of event that was decoded and a value representing that event. The currently-supported event types are "offset_commit" and "group_metadata".

For example:

local script = {}

function script.transform(record)

local event_type, data = kafka.parse_committed_offset(record)

if event_type == nil then

return record

end

record.value = tostring(data) -- or json.encode(data)

return record

end

return script

kafka.record_size(record)

number

Returns the size in bytes of the given record.

math.abs(n)

number

Returns the absolute value of n.

math.acos(n)

number

Returns the arc cosine of n in radians.

math.asin(n)

number

Returns the arc sine of n in radians.

math.atan(y, x)

number

Returns the arc tangent of y/x. The x argument defaults to 1 if not provided.

math.ceil(n)

number

Rounds n towards positive infinity.

math.cos(n)

number

Returns the cosine of n.

math.deg(n)

number

Converts the angle n from radians to degrees.

math.exp(n)

number

Raises the base of natural logarithms to n (e^n).

math.floor(n)

number

Rounds n towards negative infinity.

math.log(n, base)

number

Returns the logarithm of x in the given base. The base argument defaults to e.

math.max(n, ...)

number

Returns the largest number amongst the given set.

math.min(n, ...)

number

Returns the smallest number amongst the given set.

math.rad(n)

number

Converts the angle n from degrees to radians.

math.random(m, n)

number

With no arguments, returns a random number in the range [0.0, 1.0]. With one argument, returns a random number in the range [0.0, m]. With two arguments, returns a random number in the range [m, n].

math.randomseed(x, y)

number

With no arguments, seeds the random number generator arbitrarily. With one argument, seeds the random number generator to x. With two arguments, seeds the random number generator to x ~ y & 0x7FFFFFFF.

math.sin(n)

number

Returns the sine of n.

math.sqrt(n)

number

Returns the square root of n.

math.tan(n)

number

Returns the tangent of n.

math.tointeger(v)

number

Converts v to an integer. Returns false if the value cannot be converted.

msgpack.unpack(str)

any

Decodes the MessagePack-encoded value represented by str to Lua. Arrays and maps are represented as Lua tables. Strings and binary data are represented as Lua strings. Nil is represented as Lua nil.

See Decoding MessagePack Data for an example.

os.clock()

number

Returns the number of milliseconds the process has been running for.

os.getenv(name)

string

Returns the value of the environment variable named name.

os.remove(path)

void

Removes the file found at the given path.

os.rename(src, dst)

void

Moves the file at src to dst.

os.time()

number

Returns the current number of seconds since the UNIX epoch.

os.tmpname()

string

Creates a new temporary file and returns its path.

print(...)

void

Prints ... to standard output, separating the elements with tabs.

string.byte(str, i, j)

table

Returns the bytes in str between i and j. The i argument defaults to 1 and the j argument defaults to the length of str.

string.char(...)

string

Constructs a string from the given bytes.

string.format(fmt, ...)

string

Formats the variable arguments according to fmt. Behaves the same as the standard C function sprintf, except that there is an additional conversion specifier, %q, which quotes literal Lua values.

string.len(str)

number

Returns the length of str.

string.lower(str)

string

Returns a new string with the characters in str lowercased according to the current locale.

string.rep(str, n, sep)

string

Repeats str n times, interspersing sep between repetitions. The sep argument defaults to "".

string.reverse(str)

string

Returns a new string with the characters in str in reverse order.

string.sub(str, i, j)

string

Returns a substring of str starting from i until j. The i argument defaults to 1 and the j argument defaults to the length of str.

string.upper(str)

string

Returns a new string with the characters in str uppercased according to the current locale.

table.concat(t, sep, i, j)

string

Joins the elements from i to j in the table t, separated by s, into a string. The sep argument defaults to "" if not provided. The i argument defaults to the first element in the table and the j argument to the last element.

table.insert(t, pos, value)

void

Inserts value into the table t at pos. If pos is not provided, value is inserted at the end of the table.

table.pack(...)

table

Packs the given variable set of arguments into a table.

local t = table.pack(1, 2, 3)

print(t[1]) -- prints 1

print(t[2]) -- prints 2

print(t[3]) -- prints 3

table.remove(t, pos)

any

Removes the value at pos from the table t. If not provided, pos defaults to the last position in the table.

table.sort(t, cmp)

void

Sorts the table t in place according to cmp. If not provided, cmp defaults to the < operator. If provided, cmp must be a function of two arguments that returns a boolean value representing whether the first argument is less than the second.

table.unpack(t)

...

Unpacks the given table.

print(string.format("%d %d %s", table.unpack({1, 2, "hello"})))

-- is equivalent to --

print(string.format("%d %d %s", 1, 2, "hello"))

tostring(any)

string

Converts the argument to a string.

tonumber(str)

number

Converts the argument to a number. If the argument cannot be converted to a number, returns false.

Timestamp(seconds, localtime)

Timestamp

Returns an instance of a Timestamp. Both arguments are optional.

If provided, the timestamp argument represents seconds since the UNIX epoch (such as the values returned by os.time). The second argument determines whether the given timestamp is interpreted using the local timezone. The second argument defaults to true if not provided or if nil.

Timestamp.at(year, month, day, hour, minute, second, localtime)

Timestamp

Creates a new timestamp at the given instant. The localtime argument defaults to true.

Timestamp.fromisostring(str)

Timestamp

Parses the given ISO8601-formatted timestamp.

Timestamp:tolocal()

Timestamp

Returns a new timestamp at the same instant as this timestamp, but set to local time.

Timestamp:toutc()

Timestamp

Returns a new timestamp at the same instant as this timestamp, but set to UTC.

Timestamp:isostring()

string

Formats this timestamp using IS8601.

5.1 Renderers🔗

The bindings documented in this section can be used to render aggregated data to a window when applying a script.

render.AreaChart(xlabel, ylabel)

Chart

Returns an instance of an area chart renderer. The first argument represents the x axis label and the second argument, the y axis label. The values of the x axis must be render.Timestamps or strings and the y axis values must be numbers.

render.BarChart(xlabel, ylabel)

Chart

Returns an instance of a bar chart renderer. The first argument represents the x axis label and the second argument, the y axis.

render.CandlestickChart(xlabel, ylabel)

Chart

Returns an instance of a candlestick chart renderer. The first argument represents the x axis label and the second argument, the y axis. The values of the x axis must be render.Timestamps and the y axis values must be render.Candlesticks.

render.CandlestickChart:setwidth(width)

Chart

On Windows and Linux, you must set the width of the candles in the chart. The width of a candle must be the length of the interval (in seconds) between x ticks on the chart. For example, if the chart is to display daily candles, the width of each candle would be 86400.

On macOS, this method controls the pixel width of the candles. Use it to make thinner candles when necessary.

render.Candlestick(o, h, l, c)

Candlestick

Returns an instance of a candlestick. The arguments must be numbers representing the open, high, low and close price, respectively, of some asset.

For use on the y axis of render.CandlestickCharts.

render.Timestamp(seconds, localtime)

Timestamp

An alias for Timestamp.

render.LineChart(xlabel, ylabel)

Chart

Returns an instance of a line chart renderer. The first argument represents the x axis label and the second argument, the y axis.

render.ScatterChart(xlabel, ylabel)

Chart

Returns an instance of a scatter chart renderer. The first argument represents the x axis label and the second argument, the y axis.

render.Table(columns, ...)

Table

Returns an instance of a table renderer. The first argument is the set of columns and the variadic arguments represent the rows. For example:

function script.render(state)

return render.Table(

{"a", "b"},

{"1", "2"},

{"3", "4"}

)

end

render.HStack(...)

Stack

Returns an instance of a view that renders its children horizontally.

-- Assuming `state' looks like:

--   {

--     n  = number,

--     xs = {...},

--     ys = {...}

--   }

function script.render(state)

local total = 0

local pairs = {}

for i, age in ipairs(state.ys) do

total = total + age

pairs[i] = { x = state.xs[i]; y = age }

end

return render.VStack(

render.HStack(

string.format("Total Records: %d", state.n),

render.Table(

{"min", "max", "avg"},

{

math.min(table.unpack(state.ys)),

math.max(table.unpack(state.ys)),

total / state.n

}

)

),

render.BarChart("Name", "Age")

:setvalues(table.unpack(pairs))

:setyscale(0, 105)

:sort()

)

end

render.VStack(...)

Stack

Returns an instance of a view that renders its children vertically. See render.HStack for a usage example.

5.1.1 Charts🔗

These methods are available on all charts. Unless otherwise specified, the return value of every method is the chart itself.

See Rendering a Bar Chart for a usage example.

Chart:push(x, y)

Chart

Pushes a new entry to the end of the chart. The first argument is the x value and the second, the y value. Values may be numbers, strings, render.Timestamps or render.Candlesticks, but the types should be internally consistent per axis.

Chart:setvalues(...)

Chart

Replaces all the chart values with the given set of tables. Each given value must be a table with x and y fields.

Chart:setxscale(lo, hi)

Chart

Sets the scale of the x axis. Undefined when the x values are strings.

Chart:setyscale(lo, hi)

Chart

Sets the scale of the y axis. Undefined when the y values are strings.

Chart:sort(cmp)

Chart

Sorts the data contained by the chart according to the given cmp function. If not provided, cmp defaults to a function that compares x values using the < operator. The arguments to cmp are two tables with one field for the x and y values each.

5.2 Examples🔗

5.2.1 Decoding JSON Data🔗

Use json.decode to decode your data.

local script = {}

function script.transform(record)

local object = json.decode(record.value)

record.value = tostring(object.field)

return record

end

return script

5.2.2 Decoding Avro Data🔗

Use avro.parse to convert an Avro Schema into a codec. Then, use that codec to decode your record data.

local script = {}

local schema = [[

{

"type": "record",

"name": "Person",

"fields": [

{

"name": "Name",

"type": "string"

},

{

"name": "Age",

"type": "int"

}

]

}

]]

local person_codec = avro.parse(schema)

function script.transform(record)

local person = person_codec:read(record.value)

record.value = person.Name

return record

end

return script

You can write your schema as a Lua table and convert it to JSON using json.encode. For example, you could rewrite the above example to:

local script = {}

local schema = json.encode(

{

type = "record",

name = "Person",

fields = {

{ name = "Name", type = "string" },

{ name = "Age",  type = "int" }

}

}

)

local person_codec = avro.parse(schema)

function script.transform(record)

local person = person_codec:read(record.value)

record.value = person.Name

return record

end

return script

See this YouTube video for a live demo.

5.2.3 Decoding MessagePack Data🔗

Use msgpack.unpack to decode your data.

local script = {}

function script.transform(record)

local object = msgpack.unpack(record.value)

record.value = tostring(object.field)

return record

end

return script

5.2.4 Rendering a Bar Chart🔗

The script below renders a bar chart with record offsets on the x axis and value lengths on the y axis when applied to some already-loaded data.

local script = {}

function script.transform(record)

return record

end

function script.reduce(record, state)

state = state or {}

table.insert(state, {

x = tostring(record.offset),

y = #record.value

})

return state

end

function script.render(state)

local function cmp(a, b)

return a.x < b.x

end

return render.BarChart("offset", "length")

:setvalues(table.unpack(state))

:sort(cmp)

end

return script

See this YouTube video for a live demo of chart rendering.

6 Guides🔗

6.1 Mutual TLS🔗

Franz supports connecting to Kafka servers with mTLS (also known as “two-way SSL”) enabled. To connect to a server with mTLS, merely provide an SSL Key and an SSL Certificate during connection setup.

6.1.1 How to extract Java Keystore keys & certificates🔗

If your client key and certificates are stored in a Java Keystore file (typically, a file with the .jks extension), then you must first extract and convert them to PEM format. You can do this using the keytool utility provided by your Java Runtime Environment. For example, assuming you have a keystore file named "client.jks", you can run the following command:

The result is a PKCS12 store named "client.p12". Next, convert this store to PEM format by running:

Finally, select the "client.pem" file as both the SSL Key and the SSL Cert from the Franz Connection Dialog and connect to your broker.

7 Advanced Topics🔗

7.1 Buffering🔗

When Franz loads topic data into the Records Table, it issues one request per partition. The maximum amount requested per partition is controlled by the request size setting. Once it has received data from all topics, it sorts the data according to the sort setting and discards any data that doesn’t fit within the buffer size setting. When data is discarded, the Record Table’s status bar contains the “truncated” string.

The default request size is 1MiB, and the default buffer size is 2MiB. The default sort is “descending”, meaning that older records are discarded before newer ones. These settings can be changed on a per-topic basis from the “Options...” popover in the Records Table.

8 Privacy🔗

Apart from when checking for updates, Franz never phones home for any reason. Automatic Updates can be turned off from the Preferences Window (⌘ ,).

9 Credits🔗

Franz is built using the Racket programming language and distributes its runtime alongside the application. Racket is licensed under the MIT License.

The source code for Franz is available for all to read on GitHub.