Bloblang Methods

Methods provide most of the power in Bloblang as they allow you to augment values and can be added to any expression (including other methods):

root.doc.id = this.thing.id.string().catch(uuid_v4())
root.doc.reduced_nums = this.thing.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})
root.has_good_taste = ["pikachu","mewtwo","magmar"].contains(this.user.fav_pokemon)

# In:  {"thing":{"id":123,"nums":[5,12,18,7,25]},"user":{"fav_pokemon":"pikachu"}}

Methods support both named and nameless style arguments:

root.foo_one = this.(bar | baz).trim().replace_all(old: "dog", new: "cat")
root.foo_two = this.(bar | baz).trim().replace_all("dog", "cat")

# In:  {"bar":"  I love my dog  "}

General

apply

Apply a declared mapping to a target value.

Parameters

Name Type Description

mapping

string

The mapping to apply.

Examples

map thing {
  root.inner = this.first
}

root.foo = this.doc.apply("thing")

# In:  {"doc":{"first":"hello world"}}
# Out: {"foo":{"inner":"hello world"}}
map create_foo {
  root.name = "a foo"
  root.purpose = "to be a foo"
}

root = this
root.foo = null.apply("create_foo")

# In:  {"id":"1234"}
# Out: {"foo":{"name":"a foo","purpose":"to be a foo"},"id":"1234"}

catch

If the result of a target query fails (due to incorrect types, failed parsing, etc) the argument is returned instead.

Parameters

Name Type Description

fallback

query expression

A value to yield, or query to execute, if the target query fails.

Examples

root.doc.id = this.thing.id.string().catch(uuid_v4())

The fallback argument can be a mapping, allowing you to capture the error string and yield structured data back:

root.url = this.url.parse_url().catch(err -> {"error":err,"input":this.url})

# In:  {"url":"invalid %&# url"}
# Out: {"url":{"error":"field `this.url`: parse \"invalid %&\": invalid URL escape \"%&\"","input":"invalid %&# url"}}

When the input document is not structured attempting to reference structured fields with this will result in an error. Therefore, a convenient way to delete non-structured data is with a catch:

root = this.catch(deleted())

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":{"foo":"bar"}}

# In:  not structured data
# Out: <Message deleted>

from

Modifies a target query such that certain functions are executed from the perspective of another message in the batch. This allows you to mutate events based on the contents of other messages. Functions that support this behavior are content, json and meta.

Parameters

Name Type Description

index

integer

The message index to use as a perspective.

Examples

For example, the following map extracts the contents of the JSON field foo specifically from message index 1 of a batch, effectively overriding the field foo for all messages of a batch to that of message 1:

root = this
root.foo = json("foo").from(1)

from_all

Modifies a target query such that certain functions are executed from the perspective of each message in the batch, and returns the set of results as an array. Functions that support this behavior are content, json and meta.

Examples

root = this
root.foo_summed = json("foo").from_all().sum()

map

Executes a query on the target value, allowing you to transform or extract data from the current context.

Parameters

Name Type Description

query

query expression

A query to execute on the target.

not

Returns the logical NOT (negation) of a boolean value. Converts true to false and false to true.

or

If the result of the target query fails or resolves to null, returns the argument instead. This is an explicit method alternative to the coalesce pipe operator |.

Parameters

Name Type Description

fallback

query expression

A value to yield, or query to execute, if the target query fails or resolves to null.

Examples

root.doc.id = this.thing.id.or(uuid_v4())

Encoding and encryption

compress

Compresses a string or byte array using the specified compression algorithm. Returns compressed data as bytes. Useful for reducing payload size before transmission or storage.

Parameters

Name Type Description

algorithm

string

The compression algorithm: flate, gzip, pgzip (parallel gzip), lz4, snappy, zlib, or zstd.

level

integer

Compression level (default: -1 for default compression). Higher values increase compression ratio but use more CPU. Range and effect varies by algorithm.

Examples

Compress and encode for safe transmission:

root.compressed = content().bytes().compress("gzip").encode("base64")

# In:  {"message":"hello world I love space"}
# Out: {"compressed":"H4sIAAAJbogA/wAmANn/eyJtZXNzYWdlIjoiaGVsbG8gd29ybGQgSSBsb3ZlIHNwYWNlIn0DAHEvdwomAAAA"}

Compare compression ratios across algorithms:

root.original_size = content().length()
root.gzip_size = content().compress("gzip").length()
root.lz4_size = content().compress("lz4").length()

# In:  The quick brown fox jumps over the lazy dog. The quick brown fox jumps over the lazy dog.
# Out: {"gzip_size":114,"lz4_size":85,"original_size":89}

decode

Decodes an encoded string according to a chosen scheme.

Parameters

Name Type Description

scheme

string

The decoding scheme to use.

Examples

root.decoded = this.value.decode("hex").string()

# In:  {"value":"68656c6c6f20776f726c64"}
# Out: {"decoded":"hello world"}
root = this.encoded.decode("ascii85")

# In:  {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}
# Out: this is totally unstructured data

decompress

Decompresses a byte array using the specified decompression algorithm. Returns decompressed data as bytes. Use with data that was previously compressed using the corresponding algorithm.

Parameters

Name Type Description

algorithm

string

The decompression algorithm: gzip, pgzip (parallel gzip), zlib, bzip2, flate, snappy, lz4, or zstd.

Examples

Decompress base64-encoded compressed data:

root = this.compressed.decode("base64").decompress("gzip")

# In:  {"compressed":"H4sIAN12MWkAA8tIzcnJVyjPL8pJUfBUyMkvS1UoLkhMTgUAQpDxbxgAAAA="}
# Out: hello world I love space

Convert decompressed bytes to string for JSON output:

root.message = this.compressed.decode("base64").decompress("gzip").string()

# In:  {"compressed":"H4sIAN12MWkAA8tIzcnJVyjPL8pJUfBUyMkvS1UoLkhMTgUAQpDxbxgAAAA="}
# Out: {"message":"hello world I love space"}

decrypt_aes

Decrypts an AES-encrypted string or byte array.

Parameters

Name Type Description

scheme

string

The scheme to use for decryption, one of ctr, gcm, ofb, cbc.

key

string

A key to decrypt with.

iv

string

An initialization vector / nonce.

Examples

let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.decrypted = this.value.decode("hex").decrypt_aes("ctr", $key, $vector).string()

# In:  {"value":"84e9b31ff7400bdf80be7254"}
# Out: {"decrypted":"hello world!"}

encode

Encodes a string or byte array according to a chosen scheme.

Parameters

Name Type Description

scheme

string

The encoding scheme to use.

Examples

root.encoded = this.value.encode("hex")

# In:  {"value":"hello world"}
# Out: {"encoded":"68656c6c6f20776f726c64"}
root.encoded = content().encode("ascii85")

# In:  this is totally unstructured data
# Out: {"encoded":"FD,B0+DGm>FDl80Ci\"A>F`)8BEckl6F`M&(+Cno&@/"}

encrypt_aes

Encrypts a string or byte array using AES encryption.

Parameters

Name Type Description

scheme

string

The scheme to use for encryption, one of ctr, gcm, ofb, cbc.

key

string

A key to encrypt with.

iv

string

An initialization vector / nonce.

Examples

let key = "2b7e151628aed2a6abf7158809cf4f3c".decode("hex")
let vector = "f0f1f2f3f4f5f6f7f8f9fafbfcfdfeff".decode("hex")
root.encrypted = this.value.encrypt_aes("ctr", $key, $vector).encode("hex")

# In:  {"value":"hello world!"}
# Out: {"encrypted":"84e9b31ff7400bdf80be7254"}

hash

Hashes a string or byte array using a specified algorithm.

Parameters

Name Type Description

algorithm

string

The hashing algorithm to use.

key (optional)

string

An optional key to use.

polynomial

string

An optional polynomial key to use when selecting the crc32 algorithm, otherwise ignored. Options are IEEE (default), Castagnoli and Koopman

Examples

root.h1 = this.value.hash("sha1").encode("hex")
root.h2 = this.value.hash("hmac_sha1","static-key").encode("hex")

# In:  {"value":"hello world"}
# Out: {"h1":"2aae6c35c94fcfb415dbe95f408b9ce91ee846ed","h2":"d87e5f068fa08fe90bb95bc7c8344cb809179d76"}

The crc32 algorithm supports options for the polynomial:

root.h1 = this.value.hash(algorithm: "crc32", polynomial: "Castagnoli").encode("hex")
root.h2 = this.value.hash(algorithm: "crc32", polynomial: "Koopman").encode("hex")

# In:  {"value":"hello world"}
# Out: {"h1":"c99465aa","h2":"df373d3c"}

uuid_v5

Generates a version 5 UUID from a namespace and name.

Parameters

Name Type Description

ns (optional)

string

An optional namespace name or UUID. It supports the dns, url, oid and x500 predefined namespaces and any valid RFC-9562 UUID. If empty, the nil UUID will be used.

Examples

root.id = "example".uuid_v5()
root.id = "example".uuid_v5("x500")
root.id = "example".uuid_v5("77f836b7-9f61-46c0-851e-9b6ca3535e69")

GeoIP

geoip_anonymous_ip

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the anonymous IP associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_asn

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the ASN associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_city

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the city associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_connection_type

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the connection type associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_country

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the country associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_domain

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the domain associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_enterprise

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the enterprise associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

geoip_isp

Looks up an IP address against a MaxMind database file and, if found, returns an object describing the ISP associated with it.

Parameters

Name Type Description

path

string

A path to an mmdb (maxmind) file.

JSON web tokens

parse_jwt_es256

Parses a claims object from a JWT string encoded with ES256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es256("""-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEGtLqIBePHmIhQcf0JLgc+F/4W/oI
dp0Gta53G35VerNDgUUXmp78J2kfh4qLdh0XtmOMI587tCaqjvDAXfs//w==
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.GIRajP9JJbpTlqSCdNEz4qpQkRvzX4Q51YnTwVyxLDM9tKjR_a8ggHWn9CWj7KG0x8J56OWtmUxn112SRTZVhQ"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_es384

Parses a claims object from a JWT string encoded with ES384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es384("""-----BEGIN PUBLIC KEY-----
MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAERoz74/B6SwmLhs8X7CWhnrWyRrB13AuU
8OYeqy0qHRu9JWNw8NIavqpTmu6XPT4xcFanYjq8FbeuM11eq06C52mNmS4LLwzA
2imlFEgn85bvJoC3bnkuq4mQjwt9VxdH
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.H2HBSlrvQBaov2tdreGonbBexxtQB-xzaPL4-tNQZ6TVh7VH8VBcSwcWHYa1lBAHmdsKOFcB2Wk0SB7QWeGT3ptSgr-_EhDMaZ8bA5spgdpq5DsKfaKHrd7DbbQlmxNq"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_es512

Parses a claims object from a JWT string encoded with ES512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The ES512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_es512("""-----BEGIN PUBLIC KEY-----
MIGbMBAGByqGSM49AgEGBSuBBAAjA4GGAAQAkHLdts9P56fFkyhpYQ31M/Stwt3w
vpaxhlfudxnXgTO1IP4RQRgryRxZ19EUzhvWDcG3GQIckoNMY5PelsnCGnIBT2Xh
9NQkjWF5K6xS4upFsbGSAwQ+GIyyk5IPJ2LHgOyMSCVh5gRZXV3CZLzXujx/umC9
UeYyTt05zRRWuD+p5bY=
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.ACrpLuU7TKpAnncDCpN9m85nkL55MJ45NFOBl6-nEXmNT1eIxWjiP4pwWVbFH9et_BgN14119jbL_KqEJInPYc9nAXC6dDLq0aBU-dalvNl4-O5YWpP43-Y-TBGAsWnbMTrchILJ4-AEiICe73Ck5yWPleKg9c3LtkEFWfGs7BoPRguZ"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs256

Parses a claims object from a JWT string encoded with HS256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs256("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.YwXOM8v3gHVWcQRRRQc_zDlhmLnM62fwhFYGpiA0J1A"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs384

Parses a claims object from a JWT string encoded with HS384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs384("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.2Y8rf_ijwN4t8hOGGViON_GrirLkCQVbCOuax6EoZ3nluX0tCGezcJxbctlIfsQ2"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_hs512

Parses a claims object from a JWT string encoded with HS512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The HS512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_hs512("""dont-tell-anyone""")

# In:  {"signed":"eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.utRb0urG6LGGyranZJVo5Dk0Fns1QNcSUYPN0TObQ-YzsGGB8jrxHwM5NAJccjJZzKectEUqmmKCaETZvuX4Fg"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs256

Parses a claims object from a JWT string encoded with RS256. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS256 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs256("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.b0lH3jEupZZ4zoaly4Y_GCvu94HH6UKdKY96zfGNsIkPZpQLHIkZ7jMWlLlNOAd8qXlsBGP_i8H2qCKI4zlWJBGyPZgxXDzNRPVrTDfFpn4t4nBcA1WK2-ntXP3ehQxsaHcQU8Z_nsogId7Pme5iJRnoHWEnWtbwz5DLSXL3ZZNnRdrHM9MdI7QSDz9mojKDCaMpGN9sG7Xl-tGdBp1XzXuUOzG8S03mtZ1IgVR1uiBL2N6oohHIAunk8DIAmNWI-zgycTgzUGU7mvPkKH43qO8Ua1-13tCUBKKa8VxcotZ67Mxm1QAvBGoDnTKwWMwghLzs6d6WViXQg6eWlJcpBA"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs384

Parses a claims object from a JWT string encoded with RS384. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS384 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs384("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.orcXYBcjVE5DU7mvq4KKWFfNdXR4nEY_xupzWoETRpYmQZIozlZnM_nHxEk2dySvpXlAzVm7kgOPK2RFtGlOVaNRIa3x-pMMr-bhZTno4L8Hl4sYxOks3bWtjK7wql4uqUbqThSJB12psAXw2-S-I_FMngOPGIn4jDT9b802ottJSvTpXcy0-eKTjrV2PSkRRu-EYJh0CJZW55MNhqlt6kCGhAXfbhNazN3ASX-dmpd_JixyBKphrngr_zRA-FCn_Xf3QQDA-5INopb4Yp5QiJ7UxVqQEKI80X_JvJqz9WE1qiAw8pq5-xTen1t7zTP-HT1NbbD3kltcNa3G8acmNg"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

parse_jwt_rs512

Parses a claims object from a JWT string encoded with RS512. This method does not validate JWT claims.

Parameters

Name Type Description

signing_secret

string

The RS512 secret that was used for signing the token.

Examples

root.claims = this.signed.parse_jwt_rs512("""-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAs/ibN8r68pLMR6gRzg4S
8v8l6Q7yi8qURjkEbcNeM1rkokC7xh0I4JVTwxYSVv/JIW8qJdyspl5NIfuAVi32
WfKvSAs+NIs+DMsNPYw3yuQals4AX8hith1YDvYpr8SD44jxhz/DR9lYKZFGhXGB
+7NqQ7vpTWp3BceLYocazWJgusZt7CgecIq57ycM5hjM93BvlrUJ8nQ1a46wfL/8
Cy4P0et70hzZrsjjN41KFhKY0iUwlyU41yEiDHvHDDsTMBxAZosWjSREGfJL6Mfp
XOInTHs/Gg6DZMkbxjQu6L06EdJ+Q/NwglJdAXM7Zo9rNELqRig6DdvG5JesdMsO
+QIDAQAB
-----END PUBLIC KEY-----""")

# In:  {"signed":"eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.rsMp_X5HMrUqKnZJIxo27aAoscovRA6SSQYR9rq7pifIj0YHXxMyNyOBDGnvVALHKTi25VUGHpfNUW0VVMmae0A4t_ObNU6hVZHguWvetKZZq4FZpW1lgWHCMqgPGwT5_uOqwYCH6r8tJuZT3pqXeL0CY4putb1AN2w6CVp620nh3l8d3XWb4jaifycd_4CEVCqHuWDmohfug4VhmoVKlIXZkYoAQowgHlozATDssBSWdYtv107Wd2AzEoiXPu6e3pflsuXULlyqQnS4ELEKPYThFLafh1NqvZDPddqozcPZ-iODBW-xf3A4DYDdivnMYLrh73AZOGHexxu8ay6nDA"}
# Out: {"claims":{"iat":1516239022,"mood":"Disdainful","sub":"1234567890"}}

sign_jwt_es256

Hash and sign an object representing JSON Web Token (JWT) claims using ES256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es256("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.-8LrOdkEiv_44ADWW08lpbq41ZmHCel58NMORPq1q4Dyw0zFhqDVLrRoSvCvuyyvgXAFb9IHfR-9MlJ_2ShA9A"}
root.signed = this.claims.sign_jwt_es256(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_es384

Hash and sign an object representing JSON Web Token (JWT) claims using ES384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es384("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJFUzM4NCIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMTIzIn0.8FmTKH08dl7dyxrNu0rmvhegiIBCy-O9cddGco2e9lpZtgv5mS5qHgPkgBC5eRw1d7SRJsHwHZeehzdqT5Ba7aZJIhz9ds0sn37YQ60L7jT0j2gxCzccrt4kECHnUnLw"}
root.signed = this.claims.sign_jwt_es384(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_es512

Hash and sign an object representing JSON Web Token (JWT) claims using ES512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_es512("""-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJFUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMTIzIn0.AQbEWymoRZxDJEJtKSFFG2k2VbDCTYSuBwAZyMqexCspr3If8aERTVGif8HXG3S7TzMBCCzxkcKr3eIU441l3DlpAMNfQbkcOlBqMvNBn-CX481WyKf3K5rFHQ-6wRonz05aIsWAxCDvAozI_9J0OWllxdQ2MBAuTPbPJ38OqXsYkCQs"}
root.signed = this.claims.sign_jwt_es512(signing_secret: """-----BEGIN EC PRIVATE KEY-----
... signature data ...
-----END EC PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs256

Hash and sign an object representing JSON Web Token (JWT) claims using HS256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs256("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMTIzIn0.hUl-nngPMY_3h9vveWJUPsCcO5PeL6k9hWLnMYeFbFQ"}
root.signed = this.claims.sign_jwt_hs256(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs384

Hash and sign an object representing JSON Web Token (JWT) claims using HS384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs384("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJIUzM4NCIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMTIzIn0.zGYLr83aToon1efUNq-hw7XgT20lPvZb8sYei8x6S6mpHwb433SJdXJXx0Oio8AZ"}
root.signed = this.claims.sign_jwt_hs384(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_hs512

Hash and sign an object representing JSON Web Token (JWT) claims using HS512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_hs512("""dont-tell-anyone""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJ1c2VyMTIzIn0.zBNR9o_6EDwXXKkpKLNJhG26j8Dc-mV-YahBwmEdCrmiWt5les8I9rgmNlWIowpq6Yxs4kLNAdFhqoRz3NXT3w"}
root.signed = this.claims.sign_jwt_hs512(signing_secret: """dont-tell-anyone""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs256

Hash and sign an object representing JSON Web Token (JWT) claims using RS256.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs256("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.b0lH3jEupZZ4zoaly4Y_GCvu94HH6UKdKY96zfGNsIkPZpQLHIkZ7jMWlLlNOAd8qXlsBGP_i8H2qCKI4zlWJBGyPZgxXDzNRPVrTDfFpn4t4nBcA1WK2-ntXP3ehQxsaHcQU8Z_nsogId7Pme5iJRnoHWEnWtbwz5DLSXL3ZZNnRdrHM9MdI7QSDz9mojKDCaMpGN9sG7Xl-tGdBp1XzXuUOzG8S03mtZ1IgVR1uiBL2N6oohHIAunk8DIAmNWI-zgycTgzUGU7mvPkKH43qO8Ua1-13tCUBKKa8VxcotZ67Mxm1QAvBGoDnTKwWMwghLzs6d6WViXQg6eWlJcpBA"}
root.signed = this.claims.sign_jwt_rs256(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs384

Hash and sign an object representing JSON Web Token (JWT) claims using RS384.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs384("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJSUzM4NCIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.orcXYBcjVE5DU7mvq4KKWFfNdXR4nEY_xupzWoETRpYmQZIozlZnM_nHxEk2dySvpXlAzVm7kgOPK2RFtGlOVaNRIa3x-pMMr-bhZTno4L8Hl4sYxOks3bWtjK7wql4uqUbqThSJB12psAXw2-S-I_FMngOPGIn4jDT9b802ottJSvTpXcy0-eKTjrV2PSkRRu-EYJh0CJZW55MNhqlt6kCGhAXfbhNazN3ASX-dmpd_JixyBKphrngr_zRA-FCn_Xf3QQDA-5INopb4Yp5QiJ7UxVqQEKI80X_JvJqz9WE1qiAw8pq5-xTen1t7zTP-HT1NbbD3kltcNa3G8acmNg"}
root.signed = this.claims.sign_jwt_rs384(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

sign_jwt_rs512

Hash and sign an object representing JSON Web Token (JWT) claims using RS512.

Parameters

Name Type Description

signing_secret

string

The secret to use for signing the token.

headers (optional)

unknown

Optional object of JWT header fields to include in the token. Keys "alg", "typ", "jku", "jwk", "x5u", "x5c", "x5t","x5t#S256" and "crit" will be ignored if provided.

Examples

root.signed = this.claims.sign_jwt_rs512("""-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""")

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"eyJhbGciOiJSUzUxMiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1MTYyMzkwMjIsIm1vb2QiOiJEaXNkYWluZnVsIiwic3ViIjoiMTIzNDU2Nzg5MCJ9.rsMp_X5HMrUqKnZJIxo27aAoscovRA6SSQYR9rq7pifIj0YHXxMyNyOBDGnvVALHKTi25VUGHpfNUW0VVMmae0A4t_ObNU6hVZHguWvetKZZq4FZpW1lgWHCMqgPGwT5_uOqwYCH6r8tJuZT3pqXeL0CY4putb1AN2w6CVp620nh3l8d3XWb4jaifycd_4CEVCqHuWDmohfug4VhmoVKlIXZkYoAQowgHlozATDssBSWdYtv107Wd2AzEoiXPu6e3pflsuXULlyqQnS4ELEKPYThFLafh1NqvZDPddqozcPZ-iODBW-xf3A4DYDdivnMYLrh73AZOGHexxu8ay6nDA"}
root.signed = this.claims.sign_jwt_rs512(signing_secret: """-----BEGIN RSA PRIVATE KEY-----
... signature data ...
-----END RSA PRIVATE KEY-----""", headers: {"kid": "my-key", "x": "y"})

# In:  {"claims":{"sub":"user123"}}
# Out: {"signed":"<signed JWT token>"}

Number manipulation

abs

Returns the absolute value of an int64 or float64 number. As a special case, when an integer is provided that is the minimum value it is converted to the maximum value.

Examples

root.outs = this.ins.map_each(ele -> ele.abs())

# In:  {"ins":[9,-18,1.23,-4.56]}
# Out: {"outs":[9,18,1.23,4.56]}

bitwise_and

Performs a bitwise AND operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to AND with

Examples

root.new_value = this.value.bitwise_and(6)

# In:  {"value":12}
# Out: {"new_value":4}
root.masked = this.flags.bitwise_and(15)

# In:  {"flags":127}
# Out: {"masked":15}

bitwise_or

Performs a bitwise OR operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to OR with

Examples

root.new_value = this.value.bitwise_or(6)

# In:  {"value":12}
# Out: {"new_value":14}
root.combined = this.flags.bitwise_or(8)

# In:  {"flags":4}
# Out: {"combined":12}

bitwise_xor

Performs a bitwise XOR (exclusive OR) operation between the integer and the specified value.

Parameters

Name Type Description

value

integer

The value to XOR with

Examples

root.new_value = this.value.bitwise_xor(6)

# In:  {"value":12}
# Out: {"new_value":10}
root.toggled = this.flags.bitwise_xor(5)

# In:  {"flags":3}
# Out: {"toggled":6}

ceil

Rounds a number up to the nearest integer. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.ceil()

# In:  {"value":5.3}
# Out: {"new_value":6}

# In:  {"value":-5.9}
# Out: {"new_value":-5}
root.result = this.price.ceil()

# In:  {"price":19.99}
# Out: {"result":20}

cos

Calculates the cosine of a given angle specified in radians.

Examples

root.new_value = (this.value * (pi() / 180)).cos()

# In:  {"value":45}
# Out: {"new_value":0.7071067811865476}

# In:  {"value":0}
# Out: {"new_value":1}

# In:  {"value":180}
# Out: {"new_value":-1}

float32

Converts a numerical type into a 32-bit floating point number, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit floating point number. Please refer to the strconv.ParseFloat documentation for details regarding the supported formats.

Examples

root.out = this.in.float32()

# In:  {"in":"6.674282313423543523453425345e-11"}
# Out: {"out":6.674283e-11}

float64

Converts a numerical type into a 64-bit floating point number, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit floating point number. Please refer to the strconv.ParseFloat documentation for details regarding the supported formats.

Examples

root.out = this.in.float64()

# In:  {"in":"6.674282313423543523453425345e-11"}
# Out: {"out":6.674282313423544e-11}

floor

Rounds a number down to the nearest integer. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.floor()

# In:  {"value":5.7}
# Out: {"new_value":5}

# In:  {"value":-3.2}
# Out: {"new_value":-4}
root.whole_seconds = this.duration_seconds.floor()

# In:  {"duration_seconds":12.345}
# Out: {"whole_seconds":12}

int16

Converts a numerical type into a 16-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 16-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int16()
root.b = this.b.round().int16()
root.c = this.c.int16()
root.d = this.d.int16().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int16()

# In:  "0xDE"
# Out: 222

int32

Converts a numerical type into a 32-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int32()
root.b = this.b.round().int32()
root.c = this.c.int32()
root.d = this.d.int32().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int32()

# In:  "0xDEAD"
# Out: 57005

int64

Converts a numerical type into a 64-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int64()
root.b = this.b.round().int64()
root.c = this.c.int64()
root.d = this.d.int64().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int64()

# In:  "0xDEADBEEF"
# Out: 3735928559

int8

Converts a numerical type into a 8-bit signed integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 8-bit signed integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.int8()
root.b = this.b.round().int8()
root.c = this.c.int8()
root.d = this.d.int8().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":-12}
root = this.int8()

# In:  "0xD"
# Out: 13

log

Calculates the natural logarithm (base e) of a number.

Examples

root.new_value = this.value.log().round()

# In:  {"value":1}
# Out: {"new_value":0}

# In:  {"value":2.7183}
# Out: {"new_value":1}
root.ln_result = this.number.log()

# In:  {"number":10}
# Out: {"ln_result":2.302585092994046}

log10

Calculates the base-10 logarithm of a number.

Examples

root.new_value = this.value.log10()

# In:  {"value":100}
# Out: {"new_value":2}

# In:  {"value":1000}
# Out: {"new_value":3}
root.log_value = this.magnitude.log10()

# In:  {"magnitude":10000}
# Out: {"log_value":4}

max

Returns the largest number from an array. All elements must be numbers and the array cannot be empty.

Examples

root.biggest = this.values.max()

# In:  {"values":[0,3,2.5,7,5]}
# Out: {"biggest":7}
root.highest_temp = this.temperatures.max()

# In:  {"temperatures":[20.5,22.1,19.8,23.4]}
# Out: {"highest_temp":23.4}

min

Returns the smallest number from an array. All elements must be numbers and the array cannot be empty.

Examples

root.smallest = this.values.min()

# In:  {"values":[0,3,-2.5,7,5]}
# Out: {"smallest":-2.5}
root.lowest_temp = this.temperatures.min()

# In:  {"temperatures":[20.5,22.1,19.8,23.4]}
# Out: {"lowest_temp":19.8}

pow

Returns the number raised to the specified exponent.

Parameters

Name Type Description

exponent

float

The exponent you want to raise to the power of.

Examples

root.new_value = this.value * 10.pow(-2)

# In:  {"value":2}
# Out: {"new_value":0.02}
root.new_value = this.value.pow(-2)

# In:  {"value":2}
# Out: {"new_value":0.25}

round

Rounds a number to the nearest integer. Values at .5 round away from zero. Returns an integer if the result fits in 64-bit, otherwise returns a float.

Examples

root.new_value = this.value.round()

# In:  {"value":5.3}
# Out: {"new_value":5}

# In:  {"value":5.9}
# Out: {"new_value":6}
root.rounded = this.score.round()

# In:  {"score":87.5}
# Out: {"rounded":88}

sin

Calculates the sine of a given angle specified in radians.

Examples

root.new_value = (this.value * (pi() / 180)).sin()

# In:  {"value":45}
# Out: {"new_value":0.7071067811865475}

# In:  {"value":0}
# Out: {"new_value":0}

# In:  {"value":90}
# Out: {"new_value":1}

tan

Calculates the tangent of a given angle specified in radians.

Examples

root.new_value = "%f".format((this.value * (pi() / 180)).tan())

# In:  {"value":0}
# Out: {"new_value":"0.000000"}

# In:  {"value":45}
# Out: {"new_value":"1.000000"}

# In:  {"value":180}
# Out: {"new_value":"-0.000000"}

uint16

Converts a numerical type into a 16-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 16-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint16()
root.b = this.b.round().uint16()
root.c = this.c.uint16()
root.d = this.d.uint16().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint16()

# In:  "0xDE"
# Out: 222

uint32

Converts a numerical type into a 32-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 32-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint32()
root.b = this.b.round().uint32()
root.c = this.c.uint32()
root.d = this.d.uint32().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint32()

# In:  "0xDEAD"
# Out: 57005

uint64

Converts a numerical type into a 64-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 64-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint64()
root.b = this.b.round().uint64()
root.c = this.c.uint64()
root.d = this.d.uint64().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint64()

# In:  "0xDEADBEEF"
# Out: 3735928559

uint8

Converts a numerical type into a 8-bit unsigned integer, this is for advanced use cases where a specific data type is needed for a given component (such as the ClickHouse SQL driver).

If the value is a string then an attempt will be made to parse it as a 8-bit unsigned integer. If the target value exceeds the capacity of an integer or contains decimal values then this method will throw an error. In order to convert a floating point number containing decimals first use .round() on the value. Please refer to the strconv.ParseInt documentation for details regarding the supported formats.

Examples

root.a = this.a.uint8()
root.b = this.b.round().uint8()
root.c = this.c.uint8()
root.d = this.d.uint8().catch(0)

# In:  {"a":12,"b":12.34,"c":"12","d":-12}
# Out: {"a":12,"b":12,"c":12,"d":0}
root = this.uint8()

# In:  "0xD"
# Out: 13

Object & array manipulation

all

Tests whether all elements in an array satisfy a condition. Returns true only if the query evaluates to true for every element. Returns false for empty arrays.

Parameters

Name Type Description

test

query expression

A test query to apply to each element.

Examples

root.all_over_21 = this.patrons.all(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"all_over_21":false}

# In:  {"patrons":[{"id":"1","age":45},{"id":"2","age":23}]}
# Out: {"all_over_21":true}
root.all_positive = this.values.all(v -> v > 0)

# In:  {"values":[1,2,3,4,5]}
# Out: {"all_positive":true}

# In:  {"values":[1,-2,3,4,5]}
# Out: {"all_positive":false}

any

Tests whether at least one element in an array satisfies a condition. Returns true if the query evaluates to true for any element. Returns false for empty arrays.

Parameters

Name Type Description

test

query expression

A test query to apply to each element.

Examples

root.any_over_21 = this.patrons.any(patron -> patron.age >= 21)

# In:  {"patrons":[{"id":"1","age":18},{"id":"2","age":23}]}
# Out: {"any_over_21":true}

# In:  {"patrons":[{"id":"1","age":10},{"id":"2","age":12}]}
# Out: {"any_over_21":false}
root.has_errors = this.results.any(r -> r.status == "error")

# In:  {"results":[{"status":"ok"},{"status":"error"},{"status":"ok"}]}
# Out: {"has_errors":true}

# In:  {"results":[{"status":"ok"},{"status":"ok"}]}
# Out: {"has_errors":false}

append

Adds one or more elements to the end of an array and returns the new array. The original array is not modified.

Examples

root.foo = this.foo.append("and", "this")

# In:  {"foo":["bar","baz"]}
# Out: {"foo":["bar","baz","and","this"]}
root.combined = this.items.append(this.new_item)

# In:  {"items":["apple","banana"],"new_item":"orange"}
# Out: {"combined":["apple","banana","orange"]}

assign

Merges two objects or arrays with override behavior. For objects, source values replace destination values on key conflicts. Arrays are concatenated. To preserve both values on conflict, use the merge method instead.

Parameters

Name Type Description

with

unknown

A value to merge the target value with.

Examples

root = this.foo.assign(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":"foos","second_name":"barer"}

Override defaults with user settings:

root.config = this.defaults.assign(this.user_settings)

# In:  {"defaults":{"timeout":30,"retries":3},"user_settings":{"timeout":60}}
# Out: {"config":{"retries":3,"timeout":60}}

collapse

Flattens a nested structure into a flat object with dot-notation keys.

Parameters

Name Type Description

include_empty

bool

Whether to include empty objects and arrays in the resulting object.

Examples

root.result = this.collapse()

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.2.bar":"2"}}

Set include_empty to true to preserve empty objects and arrays in the output:

root.result = this.collapse(include_empty: true)

# In:  {"foo":[{"bar":"1"},{"bar":{}},{"bar":"2"},{"bar":[]}]}
# Out: {"result":{"foo.0.bar":"1","foo.1.bar":{},"foo.2.bar":"2","foo.3.bar":[]}}

concat

Concatenates an array value with one or more argument arrays.

Examples

root.foo = this.foo.concat(this.bar, this.baz)

# In:  {"foo":["a","b"],"bar":["c"],"baz":["d","e","f"]}
# Out: {"foo":["a","b","c","d","e","f"]}

contains

Tests if an array or object contains a value.

Parameters

Name Type Description

value

unknown

A value to test against elements of the target.

Examples

root.has_foo = this.thing.contains("foo")

# In:  {"thing":["this","foo","that"]}
# Out: {"has_foo":true}

# In:  {"thing":["this","bar","that"]}
# Out: {"has_foo":false}
root.has_bar = this.thing.contains(20)

# In:  {"thing":[10.3,20.0,"huh",3]}
# Out: {"has_bar":true}

# In:  {"thing":[2,3,40,67]}
# Out: {"has_bar":false}
root.has_foo = this.thing.contains("foo")

# In:  {"thing":"this foo that"}
# Out: {"has_foo":true}

# In:  {"thing":"this bar that"}
# Out: {"has_foo":false}

diff

Compares the current value with another value and returns a detailed changelog describing all differences. The changelog contains operations (create, update, delete) with their paths and values, enabling you to track changes between data versions, implement audit logs, or synchronize data between systems.

Parameters

Name Type Description

other

unknown

The value to compare against the current value. Can be any structured data (object or array).

Examples

Compare two objects to track field changes:

root.changes = this.before.diff(this.after)

# In:  {"before":{"name":"Alice","age":30},"after":{"name":"Alice","age":31,"city":"NYC"}}
# Out: {"changes":[{"From":30,"Path":["age"],"To":31,"Type":"update"},{"From":null,"Path":["city"],"To":"NYC","Type":"create"}]}

Detect deletions in configuration changes:

root.changelog = this.old_config.diff(this.new_config)

# In:  {"old_config":{"debug":true,"timeout":30},"new_config":{"timeout":60}}
# Out: {"changelog":[{"From":true,"Path":["debug"],"To":null,"Type":"delete"},{"From":30,"Path":["timeout"],"To":60,"Type":"update"}]}

enumerated

Transforms an array into an array of objects with index and value fields, making it easy to access both the position and content of each element.

Examples

root.foo = this.foo.enumerated()

# In:  {"foo":["bar","baz"]}
# Out: {"foo":[{"index":0,"value":"bar"},{"index":1,"value":"baz"}]}

Useful for filtering by index position:

root.first_two = this.items.enumerated().filter(item -> item.index < 2).map_each(item -> item.value)

# In:  {"items":["a","b","c","d"]}
# Out: {"first_two":["a","b"]}

exists

Checks whether a field exists at the specified dot path within an object. Returns true if the field is present (even if null), false otherwise.

Parameters

Name Type Description

path

string

A dot path to a field.

Examples

root.result = this.foo.exists("bar.baz")

# In:  {"foo":{"bar":{"baz":"yep, I exist"}}}
# Out: {"result":true}

# In:  {"foo":{"bar":{}}}
# Out: {"result":false}

# In:  {"foo":{}}
# Out: {"result":false}

Also returns true for null values if the field exists:

root.has_field = this.data.exists("optional_field")

# In:  {"data":{"optional_field":null}}
# Out: {"has_field":true}

# In:  {"data":{}}
# Out: {"has_field":false}

explode

Expands a nested field into multiple documents.

Parameters

Name Type Description

path

string

A dot path to a field to explode.

Examples

On arrays

When exploding an array, each element becomes a separate document with the array element replacing the original field:

root = this.explode("value")

# In:  {"id":1,"value":["foo","bar","baz"]}
# Out: [{"id":1,"value":"foo"},{"id":1,"value":"bar"},{"id":1,"value":"baz"}]
On objects

When exploding an object, the output keys match the nested object’s keys, with values being the full document where the target field is replaced by each nested value:

root = this.explode("value")

# In:  {"id":1,"value":{"foo":2,"bar":[3,4],"baz":{"bev":5}}}
# Out: {"bar":{"id":1,"value":[3,4]},"baz":{"id":1,"value":{"bev":5}},"foo":{"id":1,"value":2}}

filter

Filters array or object elements based on a condition.

Parameters

Name Type Description

test

query expression

A query to apply to each element, if this query resolves to any value other than a boolean true the element will be removed from the result.

Examples

root.new_nums = this.nums.filter(num -> num > 10)

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[11,17]}
On objects

When filtering objects, the query receives a context with key and value fields for each entry:

root.new_dict = this.dict.filter(item -> item.value.contains("foo"))

# In:  {"dict":{"first":"hello foo","second":"world","third":"this foo is great"}}
# Out: {"new_dict":{"first":"hello foo","third":"this foo is great"}}

find

Searches an array for a matching value and returns the index of the first occurrence. Returns -1 if no match is found. Numeric types are compared by value regardless of representation.

Parameters

Name Type Description

value

unknown

A value to find.

Examples

root.index = this.find("bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":1}
root.index = this.things.find(this.goal)

# In:  {"goal":"bar","things":["foo", "bar", "baz"]}
# Out: {"index":1}

find_all

Searches an array for all occurrences of a value and returns an array of matching indexes. Returns an empty array if no matches are found. Numeric types are compared by value regardless of representation.

Parameters

Name Type Description

value

unknown

A value to find.

Examples

root.index = this.find_all("bar")

# In:  ["foo", "bar", "baz", "bar"]
# Out: {"index":[1,3]}
root.indexes = this.things.find_all(this.goal)

# In:  {"goal":"bar","things":["foo", "bar", "baz", "bar", "buz"]}
# Out: {"indexes":[1,3]}

find_all_by

Searches an array for all elements that satisfy a condition and returns an array of their indexes. Returns an empty array if no elements match.

Parameters

Name Type Description

query

query expression

A query to execute for each element.

Examples

root.index = this.find_all_by(v -> v != "bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":[0,2]}

Find all indexes matching criteria:

root.error_indexes = this.logs.find_all_by(log -> log.level == "error")

# In:  {"logs":[{"level":"info"},{"level":"error"},{"level":"warn"},{"level":"error"}]}
# Out: {"error_indexes":[1,3]}

find_by

Searches an array for the first element that satisfies a condition and returns its index. Returns -1 if no element matches the query.

Parameters

Name Type Description

query

query expression

A query to execute for each element.

Examples

root.index = this.find_by(v -> v != "bar")

# In:  ["foo", "bar", "baz"]
# Out: {"index":0}

Find first object matching criteria:

root.first_adult = this.users.find_by(u -> u.age >= 18)

# In:  {"users":[{"name":"Alice","age":15},{"name":"Bob","age":22},{"name":"Carol","age":19}]}
# Out: {"first_adult":1}

flatten

Flattens an array by one level, expanding nested arrays into the parent array. Only the first level of nesting is removed.

Examples

root.result = this.flatten()

# In:  ["foo",["bar","baz"],"buz"]
# Out: {"result":["foo","bar","baz","buz"]}

Deeper nesting requires multiple flatten calls:

root.result = this.data.flatten()

# In:  {"data":["a",["b",["c","d"]],"e"]}
# Out: {"result":["a","b",["c","d"],"e"]}

fold

Reduces an array to a single value by iteratively applying a function. Also known as reduce or aggregate. The query receives an accumulator (tally) and current element (value) for each iteration.

Parameters

Name Type Description

initial

unknown

The initial value to start the fold with. For example, an empty object {}, a zero count 0, or an empty string "".

query

query expression

A query to apply for each element. The query is provided an object with two fields; tally containing the current tally, and value containing the value of the current element. The query should result in a new tally to be passed to the next element query.

Examples

Sum numbers in an array:

root.sum = this.foo.fold(0, item -> item.tally + item.value)

# In:  {"foo":[3,8,11]}
# Out: {"sum":22}

Concatenate strings:

root.result = this.foo.fold("", item -> "%v%v".format(item.tally, item.value))

# In:  {"foo":["hello ", "world"]}
# Out: {"result":"hello world"}

Merge an array of objects into a single object:

root.smoothie = this.fruits.fold({}, item -> item.tally.merge(item.value))

# In:  {"fruits":[{"apple":5},{"banana":3},{"orange":8}]}
# Out: {"smoothie":{"apple":5,"banana":3,"orange":8}}

get

Extract a field value, identified via a dot path, from an object.

Parameters

Name Type Description

path

string

A dot path identifying a field to obtain.

Examples

root.result = this.foo.get(this.target)

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"bar"}
# Out: {"result":"from bar"}

# In:  {"foo":{"bar":"from bar","baz":"from baz"},"target":"baz"}
# Out: {"result":"from baz"}

index

Extract an element from an array by an index. The index can be negative, and if so the element will be selected from the end counting backwards starting from -1. E.g. an index of -1 returns the last element, an index of -2 returns the element before the last, and so on.

Parameters

Name Type Description

index

integer

The index to obtain from an array.

Examples

root.last_name = this.names.index(-1)

# In:  {"names":["rachel","stevens"]}
# Out: {"last_name":"stevens"}

It is also possible to use this method on byte arrays, in which case the selected element will be returned as an integer:

root.last_byte = this.name.bytes().index(-1)

# In:  {"name":"foobar bazson"}
# Out: {"last_byte":110}

join

Joins an array of strings with an optional delimiter.

Parameters

Name Type Description

delimiter (optional)

string

An optional delimiter to add between each string.

Examples

root.joined_words = this.words.join()
root.joined_numbers = this.numbers.map_each(this.string()).join(",")

# In:  {"words":["hello","world"],"numbers":[3,8,11]}
# Out: {"joined_numbers":"3,8,11","joined_words":"helloworld"}

json_path

Executes the given JSONPath expression on an object or array and returns the result. The JSONPath expression syntax can be found at https://goessner.net/articles/JsonPath/. For more complex logic, you can use Gval expressions (https://github.com/PaesslerAG/gval).

Parameters

Name Type Description

expression

string

The JSONPath expression to execute.

Examples

root.all_names = this.json_path("$..name")

# In:  {"name":"alice","foo":{"name":"bob"}}
# Out: {"all_names":["alice","bob"]}

# In:  {"thing":["this","bar",{"name":"alice"}]}
# Out: {"all_names":["alice"]}
root.text_objects = this.json_path("$.body[?(@.type=='text')]")

# In:  {"body":[{"type":"image","id":"foo"},{"type":"text","id":"bar"}]}
# Out: {"text_objects":[{"id":"bar","type":"text"}]}

json_schema

Checks a JSON schema against a value and returns the value if it matches or throws and error if it does not.

Parameters

Name Type Description

schema

string

The schema to check values against.

Examples

root = this.json_schema("""{
  "type":"object",
  "properties":{
    "foo":{
      "type":"string"
    }
  }
}""")

# In:  {"foo":"bar"}
# Out: {"foo":"bar"}

# In:  {"foo":5}
# Out: Error("failed assignment (line 1): field `this`: foo invalid type. expected: string, given: integer")

In order to load a schema from a file use the file function:

root = this.json_schema(file(env("BENTHOS_TEST_BLOBLANG_SCHEMA_FILE")))

key_values

Converts an object into an array of key-value pair objects. Each element has a 'key' field and a 'value' field. Order is not guaranteed unless sorted.

Examples

root.foo_key_values = this.foo.key_values().sort_by(pair -> pair.key)

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_key_values":[{"key":"bar","value":1},{"key":"baz","value":2}]}

Filter object entries by value:

root.large_items = this.items.key_values().filter(pair -> pair.value > 15).map_each(pair -> pair.key)

# In:  {"items":{"a":5,"b":15,"c":20,"d":3}}
# Out: {"large_items":["c"]}

keys

Extracts all keys from an object and returns them as a sorted array.

Examples

root.foo_keys = this.foo.keys()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_keys":["bar","baz"]}

Check if specific keys exist:

root.has_id = this.data.keys().contains("id")

# In:  {"data":{"id":123,"name":"test"}}
# Out: {"has_id":true}

length

Returns the length of an array, object, or string.

Examples

root.foo_len = this.foo.length()

# In:  {"foo":"hello world"}
# Out: {"foo_len":11}
root.foo_len = this.foo.length()

# In:  {"foo":["first","second"]}
# Out: {"foo_len":2}

# In:  {"foo":{"first":"bar","second":"baz"}}
# Out: {"foo_len":2}

map_each

Applies a mapping to each element of an array or object.

Parameters

Name Type Description

query

query expression

A query that will be used to map each element.

Examples

On arrays

Transforms each array element using a query. Return deleted() to remove an element, or the new value to replace it:

root.new_nums = this.nums.map_each(num -> if num < 10 {
  deleted()
} else {
  num - 10
})

# In:  {"nums":[3,11,4,17]}
# Out: {"new_nums":[1,7]}
On objects

Transforms each object value using a query. The query receives an object with 'key' and 'value' fields for each entry:

root.new_dict = this.dict.map_each(item -> item.value.uppercase())

# In:  {"dict":{"foo":"hello","bar":"world"}}
# Out: {"new_dict":{"bar":"WORLD","foo":"HELLO"}}

map_each_key

Transforms object keys using a mapping query.

Parameters

Name Type Description

query

query expression

A query that will be used to map each key.

Examples

root.new_dict = this.dict.map_each_key(key -> key.uppercase())

# In:  {"dict":{"keya":"hello","keyb":"world"}}
# Out: {"new_dict":{"KEYA":"hello","KEYB":"world"}}

Conditionally transform keys:

root = this.map_each_key(key -> if key.contains("kafka") { "_" + key })

# In:  {"amqp_key":"foo","kafka_key":"bar","kafka_topic":"baz"}
# Out: {"_kafka_key":"bar","_kafka_topic":"baz","amqp_key":"foo"}

merge

Combines two objects or arrays. When merging objects, conflicting keys create arrays containing both values. Arrays are concatenated. For key override behavior instead, use the assign method.

Parameters

Name Type Description

with

unknown

A value to merge the target value with.

Examples

root = this.foo.merge(this.bar)

# In:  {"foo":{"first_name":"fooer","likes":"bars"},"bar":{"second_name":"barer","likes":"foos"}}
# Out: {"first_name":"fooer","likes":["bars","foos"],"second_name":"barer"}

Merge arrays:

root.combined = this.list1.merge(this.list2)

# In:  {"list1":["a","b"],"list2":["c","d"]}
# Out: {"combined":["a","b","c","d"]}

patch

Applies a changelog (created by the diff method) to the current value, transforming it according to the specified operations. This enables you to synchronize data, replay changes, or implement event sourcing patterns by applying recorded changes to reconstruct state.

Parameters

Name Type Description

changelog

unknown

The changelog array to apply. Should be in the format returned by the diff method, containing Type, Path, From, and To fields for each change.

Examples

Apply recorded changes to update an object:

root.updated = this.current.patch(this.changelog)

# In:  {"current":{"name":"Alice","age":30},"changelog":[{"Type":"update","Path":["age"],"From":30,"To":31},{"Type":"create","Path":["city"],"From":null,"To":"NYC"}]}
# Out: {"updated":{"age":31,"city":"NYC","name":"Alice"}}

Restore previous state by applying inverse changes:

root.restored = this.modified.patch(this.reverse_changelog)

# In:  {"modified":{"timeout":60},"reverse_changelog":[{"Type":"create","Path":["debug"],"From":null,"To":true},{"Type":"update","Path":["timeout"],"From":60,"To":30}]}
# Out: {"restored":{"debug":true,"timeout":30}}

slice

Extracts a portion of an array or string.

Parameters

Name Type Description

low

integer

The low bound, which is the first element of the selection, or if negative selects from the end.

high (optional)

integer

An optional high bound.

Examples

root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":"foo bar"}
# Out: {"beginning":"fo","end":"bar"}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-4)
root.the_rest = this.value.slice(0, -4)

# In:  {"value":"foo bar"}
# Out: {"last_chunk":" bar","the_rest":"foo"}
root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"beginning":["foo","bar"],"end":["bev"]}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-2)
root.the_rest = this.value.slice(0, -2)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"last_chunk":["buz","bev"],"the_rest":["foo","bar","baz"]}

sort

Sorts array elements in ascending order.

Parameters

Name Type Description

compare (optional)

query expression

An optional query that should explicitly compare elements left and right and provide a boolean result.

Examples

root.sorted = this.foo.sort()

# In:  {"foo":["bbb","ccc","aaa"]}
# Out: {"sorted":["aaa","bbb","ccc"]}

Custom comparison for complex objects - return true if left < right:

root.sorted = this.foo.sort(item -> item.left.v < item.right.v)

# In:  {"foo":[{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"},{"id":"baz","v":"aaa"}]}
# Out: {"sorted":[{"id":"baz","v":"aaa"},{"id":"foo","v":"bbb"},{"id":"bar","v":"ccc"}]}

sort_by

Sorts array elements by a specified field or expression.

Parameters

Name Type Description

query

query expression

A query to apply to each element that yields a value used for sorting.

Examples

root.sorted = this.foo.sort_by(ele -> ele.id)

# In:  {"foo":[{"id":"bbb","message":"bar"},{"id":"aaa","message":"foo"},{"id":"ccc","message":"baz"}]}
# Out: {"sorted":[{"id":"aaa","message":"foo"},{"id":"bbb","message":"bar"},{"id":"ccc","message":"baz"}]}

Sort by numeric field:

root.sorted = this.items.sort_by(item -> item.priority)

# In:  {"items":[{"name":"low","priority":3},{"name":"high","priority":1},{"name":"med","priority":2}]}
# Out: {"sorted":[{"name":"high","priority":1},{"name":"med","priority":2},{"name":"low","priority":3}]}

squash

Squashes an array of objects into a single object, where key collisions result in the values being merged (following similar rules as the .merge() method).

Examples

root.locations = this.locations.map_each(loc -> {loc.state: [loc.name]}).squash()

# In:  {"locations":[{"name":"Seattle","state":"WA"},{"name":"New York","state":"NY"},{"name":"Bellevue","state":"WA"},{"name":"Olympia","state":"WA"}]}
# Out: {"locations":{"NY":["New York"],"WA":["Seattle","Bellevue","Olympia"]}}

sum

Returns the sum of numeric values in an array.

Examples

root.sum = this.foo.sum()

# In:  {"foo":[3,8,4]}
# Out: {"sum":15}

Works with decimals:

root.total = this.prices.sum()

# In:  {"prices":[10.5,20.25,5.00]}
# Out: {"total":35.75}

unique

Returns an array with duplicate elements removed.

Parameters

Name Type Description

emit (optional)

query expression

An optional query that can be used in order to yield a value for each element to determine uniqueness.

Examples

root.uniques = this.foo.unique()

# In:  {"foo":["a","b","a","c"]}
# Out: {"uniques":["a","b","c"]}

Use a query to determine uniqueness by a field:

root.unique_users = this.users.unique(u -> u.id)

# In:  {"users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"},{"id":1,"name":"Alice Duplicate"}]}
# Out: {"unique_users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}

values

Returns an array of all values from an object.

Examples

root.foo_vals = this.foo.values().sort()

# In:  {"foo":{"bar":1,"baz":2}}
# Out: {"foo_vals":[1,2]}

Find max value in object:

root.max = this.scores.values().sort().index(-1)

# In:  {"scores":{"player1":85,"player2":92,"player3":78}}
# Out: {"max":92}

with

Returns an object where all but one or more field path arguments are removed. Each path specifies a specific field to be retained from the input object, allowing for nested fields.

If a key within a nested path does not exist then it is ignored.

Examples

root = this.with("inner.a","inner.c","d")

# In:  {"inner":{"a":"first","b":"second","c":"third"},"d":"fourth","e":"fifth"}
# Out: {"d":"fourth","inner":{"a":"first","c":"third"}}

without

Returns an object with specified keys removed.

Examples

root = this.without("inner.a","inner.c","d")

# In:  {"inner":{"a":"first","b":"second","c":"third"},"d":"fourth","e":"fifth"}
# Out: {"e":"fifth","inner":{"b":"second"}}

Remove sensitive fields:

root = this.without("password","ssn","creditCard")

# In:  {"username":"alice","password":"secret","email":"alice@example.com","ssn":"123-45-6789"}
# Out: {"email":"alice@example.com","username":"alice"}

zip

Zip an array value with one or more argument arrays. Each array must match in length.

Examples

root.foo = this.foo.zip(this.bar, this.baz)

# In:  {"foo":["a","b","c"],"bar":[1,2,3],"baz":[4,5,6]}
# Out: {"foo":[["a",1,4],["b",2,5],["c",3,6]]}

Parsing

bloblang

Executes an argument Bloblang mapping on the target. This method can be used in order to execute dynamic mappings. Imports and functions that interact with the environment, such as file and env, or that access message information directly, such as content or json, are not enabled for dynamic Bloblang mappings.

Parameters

Name Type Description

mapping

string

The mapping to execute.

Examples

root.body = this.body.bloblang(this.mapping)

# In:  {"body":{"foo":"hello world"},"mapping":"root.foo = this.foo.uppercase()"}
# Out: {"body":{"foo":"HELLO WORLD"}}

# In:  {"body":{"foo":"hello world 2"},"mapping":"root.foo = this.foo.capitalize()"}
# Out: {"body":{"foo":"Hello World 2"}}

format_json

Formats a value as a JSON string.

Parameters

Name Type Description

indent

string

Indentation string. Each element in a JSON object or array will begin on a new, indented line followed by one or more copies of indent according to the indentation nesting.

no_indent

bool

Disable indentation.

escape_html

bool

Escape problematic HTML characters.

Examples

root = this.doc.format_json()

# In:  {"doc":{"foo":"bar"}}
# Out: {
    "foo": "bar"
}

Pass a string to the indent parameter in order to customise the indentation:

root = this.format_json("  ")

# In:  {"doc":{"foo":"bar"}}
# Out: {
  "doc": {
    "foo": "bar"
  }
}

Use the .string() method in order to coerce the result into a string:

root.doc = this.doc.format_json().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"{\n    \"foo\": \"bar\"\n}"}

Set the no_indent parameter to true to disable indentation. The result is equivalent to calling bytes():

root = this.doc.format_json(no_indent: true)

# In:  {"doc":{"foo":"bar"}}
# Out: {"foo":"bar"}

Escapes problematic HTML characters:

root = this.doc.format_json()

# In:  {"doc":{"email":"foo&bar@benthos.dev","name":"foo>bar"}}
# Out: {
    "email": "foo\u0026bar@benthos.dev",
    "name": "foo\u003ebar"
}

Set the escape_html parameter to false to disable escaping of problematic HTML characters:

root = this.doc.format_json(escape_html: false)

# In:  {"doc":{"email":"foo&bar@benthos.dev","name":"foo>bar"}}
# Out: {
    "email": "foo&bar@benthos.dev",
    "name": "foo>bar"
}

format_msgpack

Serializes structured data into MessagePack binary format. MessagePack is a compact binary serialization that is faster and more space-efficient than JSON, making it ideal for network transmission and storage of structured data. Returns a byte array that can be further encoded as needed.

Examples

Serialize object to MessagePack and encode as hex for transmission:

root = this.format_msgpack().encode("hex")

# In:  {"foo":"bar"}
# Out: 81a3666f6fa3626172

Serialize data to MessagePack and base64 encode for embedding in JSON:

root.msgpack_payload = this.data.format_msgpack().encode("base64")

# In:  {"data":{"foo":"bar"}}
# Out: {"msgpack_payload":"gaNmb2+jYmFy"}

format_xml

Serializes an object into an XML document. Converts structured data to XML format with support for attributes (prefixed with hyphen), custom indentation, and configurable root element. Returns XML as a byte array.

Parameters

Name Type Description

indent

string

String to use for each level of indentation (default is 4 spaces). Each nested XML element will be indented by this string.

no_indent

bool

Disable indentation and newlines to produce compact XML on a single line.

root_tag (optional)

string

Custom name for the root XML element. By default, the root element name is derived from the first key in the object.

Examples

Serialize object to pretty-printed XML with default indentation:

root = this.format_xml()

# In:  {"foo":{"bar":{"baz":"foo bar baz"}}}
# Out: <foo>
    <bar>
        <baz>foo bar baz</baz>
    </bar>
</foo>

Create compact XML without indentation for smaller message size:

root = this.format_xml(no_indent: true)

# In:  {"foo":{"bar":{"baz":"foo bar baz"}}}
# Out: <foo><bar><baz>foo bar baz</baz></bar></foo>

format_yaml

Formats a value as a YAML string.

Examples

root = this.doc.format_yaml()

# In:  {"doc":{"foo":"bar"}}
# Out: foo: bar

Use the .string() method in order to coerce the result into a string:

root.doc = this.doc.format_yaml().string()

# In:  {"doc":{"foo":"bar"}}
# Out: {"doc":"foo: bar\n"}

infer_schema

Attempt to infer the schema of a given value. The resulting schema can then be used as an input to schema conversion and enforcement methods.

parse_csv

Parses CSV data into an array.

Parameters

Name Type Description

parse_header_row

bool

Whether to reference the first row as a header row. If set to true the output structure for messages will be an object where field keys are determined by the header row. Otherwise, the output will be an array of row arrays.

delimiter

string

The delimiter to use for splitting values in each record. It must be a single character.

lazy_quotes

bool

If set to true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field.

Examples

Parses CSV data with a header row:

root.orders = this.orders.parse_csv()

# In:  {"orders":"foo,bar\nfoo 1,bar 1\nfoo 2,bar 2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar 2","foo":"foo 2"}]}

Parses CSV data without a header row:

root.orders = this.orders.parse_csv(false)

# In:  {"orders":"foo 1,bar 1\nfoo 2,bar 2"}
# Out: {"orders":[["foo 1","bar 1"],["foo 2","bar 2"]]}

Parses CSV data delimited by dots:

root.orders = this.orders.parse_csv(delimiter:".")

# In:  {"orders":"foo.bar\nfoo 1.bar 1\nfoo 2.bar 2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar 2","foo":"foo 2"}]}

Parses CSV data containing a quote in an unquoted field:

root.orders = this.orders.parse_csv(lazy_quotes:true)

# In:  {"orders":"foo,bar\nfoo 1,bar 1\nfoo\" \"2,bar\" \"2"}
# Out: {"orders":[{"bar":"bar 1","foo":"foo 1"},{"bar":"bar\" \"2","foo":"foo\" \"2"}]}

parse_form_url_encoded

Attempts to parse a url-encoded query string (from an x-www-form-urlencoded request body) and returns a structured result.

Examples

root.values = this.body.parse_form_url_encoded()

# In:  {"body":"noise=meow&animal=cat&fur=orange&fur=fluffy"}
# Out: {"values":{"animal":"cat","fur":["orange","fluffy"],"noise":"meow"}}

parse_json

Parses a JSON string into a structured value.

Parameters

Name Type Description

use_number (optional)

bool

An optional flag that when set makes parsing numbers as json.Number instead of the default float64.

Examples

root.doc = this.doc.parse_json()

# In:  {"doc":"{\"foo\":\"bar\"}"}
# Out: {"doc":{"foo":"bar"}}
root.doc = this.doc.parse_json(use_number: true)

# In:  {"doc":"{\"foo\":\"11380878173205700000000000000000000000000000000\"}"}
# Out: {"doc":{"foo":"11380878173205700000000000000000000000000000000"}}

parse_logfmt

Parses logfmt formatted data into an object.

Examples

root = this.msg.parse_logfmt()

# In:  {"msg":"level=info msg=\"hello world\" dur=1.5s"}
# Out: {"dur":"1.5s","level":"info","msg":"hello world"}

parse_msgpack

Parses MessagePack binary data into a structured object. MessagePack is an efficient binary serialization format that is more compact than JSON while maintaining similar data structures. Commonly used for high-performance APIs and data interchange between microservices.

Examples

Parse MessagePack data from hex-encoded content:

root = content().decode("hex").parse_msgpack()

# In:  81a3666f6fa3626172
# Out: {"foo":"bar"}

Parse MessagePack from base64-encoded field:

root.decoded = this.msgpack_data.decode("base64").parse_msgpack()

# In:  {"msgpack_data":"gaNmb2+jYmFy"}
# Out: {"decoded":{"foo":"bar"}}

parse_parquet

Parses Apache Parquet binary data into an array of objects. Parquet is a columnar storage format optimized for analytics, commonly used with big data systems like Apache Spark, Hive, and cloud data warehouses. Each row in the Parquet file becomes an object in the output array.

Parameters

Name Type Description

byte_array_as_string

bool

Deprecated: This parameter is no longer used.

Examples

Parse Parquet file data into structured objects:

root.records = content().parse_parquet()

Process Parquet data from a field and extract specific columns:

root.users = this.parquet_data.parse_parquet().map_each(row -> {"name": row.name, "email": row.email})

parse_url

Attempts to parse a URL from a string value, returning a structured result that describes the various facets of the URL. The fields returned within the structured result roughly follow https://pkg.go.dev/net/url#URL, and may be expanded in future in order to present more information.

Examples

root.foo_url = this.foo_url.parse_url()

# In:  {"foo_url":"https://docs.redpanda.com/redpanda-connect/guides/bloblang/about/"}
# Out: {"foo_url":{"fragment":"","host":"docs.redpanda.com","opaque":"","path":"/redpanda-connect/guides/bloblang/about/","raw_fragment":"","raw_path":"","raw_query":"","scheme":"https"}}
root.username = this.url.parse_url().user.name | "unknown"

# In:  {"url":"amqp://foo:bar@127.0.0.1:5672/"}
# Out: {"username":"foo"}

# In:  {"url":"redis://localhost:6379"}
# Out: {"username":"unknown"}

parse_xml

Parses an XML document into a structured object. Converts XML elements to JSON-like objects following these rules:

  • Element attributes are prefixed with a hyphen (e.g., -id for an id attribute)

  • Elements with both attributes and text content store the text in a #text field

  • Repeated elements become arrays

  • XML comments, directives, and processing instructions are ignored

  • Optionally cast numeric and boolean strings to their proper types.

Parameters

Name Type Description

cast (optional)

bool

Whether to automatically cast numeric and boolean string values to their proper types. When false, all values remain as strings.

Examples

Parse XML document into object structure:

root.doc = this.doc.parse_xml()

# In:  {"doc":"<root><title>This is a title</title><content>This is some content</content></root>"}
# Out: {"doc":{"root":{"content":"This is some content","title":"This is a title"}}}

Parse XML with type casting enabled to convert strings to numbers and booleans:

root.doc = this.doc.parse_xml(cast: true)

# In:  {"doc":"<root><title>This is a title</title><number id=\"99\">123</number><bool>True</bool></root>"}
# Out: {"doc":{"root":{"bool":true,"number":{"#text":123,"-id":99},"title":"This is a title"}}}

parse_yaml

Parses a YAML string into a structured value.

Examples

root.doc = this.doc.parse_yaml()

# In:  {"doc":"foo: bar"}
# Out: {"doc":{"foo":"bar"}}

Regular expressions

re_find_all

Finds all matches of a regular expression in a string.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all("a.")

# In:  {"value":"paranormal"}
# Out: {"matches":["ar","an","al"]}
root.numbers = this.text.re_find_all("[0-9]+")

# In:  {"text":"I have 2 apples and 15 oranges"}
# Out: {"numbers":["2","15"]}

re_find_all_object

Finds all regex matches as objects with named groups.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[{"0":"axxb","foo":"xx"},{"0":"ab","foo":""}]}
root.matches = this.value.re_find_all_object("(?m)(?P<key>\\w+):\\s+(?P<value>\\w+)$")

# In:  {"value":"option1: value1\noption2: value2\noption3: value3"}
# Out: {"matches":[{"0":"option1: value1","key":"option1","value":"value1"},{"0":"option2: value2","key":"option2","value":"value2"},{"0":"option3: value3","key":"option3","value":"value3"}]}

re_find_all_submatch

Finds all regex matches with capture groups.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_all_submatch("a(x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":[["axxb","xx"],["ab",""]]}
root.emails = this.text.re_find_all_submatch("(\\w+)@(\\w+\\.\\w+)")

# In:  {"text":"Contact: alice@example.com or bob@test.org"}
# Out: {"emails":[["alice@example.com","alice","example.com"],["bob@test.org","bob","test.org"]]}

re_find_object

Finds the first regex match as an object with named groups.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_find_object("a(?P<foo>x*)b")

# In:  {"value":"-axxb-ab-"}
# Out: {"matches":{"0":"axxb","foo":"xx"}}
root.matches = this.value.re_find_object("(?P<key>\\w+):\\s+(?P<value>\\w+)")

# In:  {"value":"option1: value1"}
# Out: {"matches":{"0":"option1: value1","key":"option1","value":"value1"}}

re_match

Tests if a string matches a regular expression.

Parameters

Name Type Description

pattern

string

The pattern to match against.

Examples

root.matches = this.value.re_match("[0-9]")

# In:  {"value":"there are 10 puppies"}
# Out: {"matches":true}

# In:  {"value":"there are ten puppies"}
# Out: {"matches":false}

re_replace

Replaces all regex matches with a replacement string that can reference capture groups using $1, $2, etc. Use for pattern-based transformations or data reformatting.

Parameters

Name Type Description

pattern

string

The pattern to match against.

value

string

The value to replace with.

re_replace_all

Replaces all regex matches with a replacement string.

Parameters

Name Type Description

pattern

string

The pattern to match against.

value

string

The value to replace with.

Examples

root.new_value = this.value.re_replace_all("ADD ([0-9]+)","+($1)")

# In:  {"value":"foo ADD 70"}
# Out: {"new_value":"foo +(70)"}
root.masked = this.email.re_replace_all("(\\w{2})\\w+@", "$1***@")

# In:  {"email":"alice@example.com"}
# Out: {"masked":"al***@example.com"}

SQL

vector

Converts an array of numbers into a vector type suitable for insertion into SQL databases with vector/embedding support. This is commonly used with PostgreSQL’s pgvector extension for storing and querying machine learning embeddings, enabling similarity search and vector operations in your database.

Examples

Convert embeddings array to vector for pgvector storage:

root.embedding = this.embeddings.vector()
root.text = this.text

Process ML model output into database-ready vector format:

root.doc_id = this.id
root.vector_embedding = this.model_output.map_each(num -> num.number()).vector()

String manipulation

capitalize

Converts a string to title case with Unicode letter mapping.

Examples

root.title = this.title.capitalize()

# In:  {"title":"the foo bar"}
# Out: {"title":"The Foo Bar"}
root.name = this.name.capitalize()

# In:  {"name":"alice smith"}
# Out: {"name":"Alice Smith"}

compare_argon2

Checks whether a string matches a hashed secret using Argon2.

Parameters

Name Type Description

hashed_secret

string

The hashed secret to compare with the input. This must be a fully-qualified string which encodes the Argon2 options used to generate the hash.

Examples

root.match = this.secret.compare_argon2("$argon2id$v=19$m=4096,t=3,p=1$c2FsdHktbWNzYWx0ZmFjZQ$RMUMwgtS32/mbszd+ke4o4Ej1jFpYiUqY6MHWa69X7Y")

# In:  {"secret":"there-are-many-blobs-in-the-sea"}
# Out: {"match":true}
root.match = this.secret.compare_argon2("$argon2id$v=19$m=4096,t=3,p=1$c2FsdHktbWNzYWx0ZmFjZQ$RMUMwgtS32/mbszd+ke4o4Ej1jFpYiUqY6MHWa69X7Y")

# In:  {"secret":"will-i-ever-find-love"}
# Out: {"match":false}

compare_bcrypt

Checks whether a string matches a hashed secret using bcrypt.

Parameters

Name Type Description

hashed_secret

string

The hashed secret value to compare with the input.

Examples

root.match = this.secret.compare_bcrypt("$2y$10$Dtnt5NNzVtMCOZONT705tOcS8It6krJX8bEjnDJnwxiFKsz1C.3Ay")

# In:  {"secret":"there-are-many-blobs-in-the-sea"}
# Out: {"match":true}
root.match = this.secret.compare_bcrypt("$2y$10$Dtnt5NNzVtMCOZONT705tOcS8It6krJX8bEjnDJnwxiFKsz1C.3Ay")

# In:  {"secret":"will-i-ever-find-love"}
# Out: {"match":false}

contains

Tests if an array or object contains a value.

Parameters

Name Type Description

value

unknown

A value to test against elements of the target.

Examples

root.has_foo = this.thing.contains("foo")

# In:  {"thing":["this","foo","that"]}
# Out: {"has_foo":true}

# In:  {"thing":["this","bar","that"]}
# Out: {"has_foo":false}
root.has_bar = this.thing.contains(20)

# In:  {"thing":[10.3,20.0,"huh",3]}
# Out: {"has_bar":true}

# In:  {"thing":[2,3,40,67]}
# Out: {"has_bar":false}
root.has_foo = this.thing.contains("foo")

# In:  {"thing":"this foo that"}
# Out: {"has_foo":true}

# In:  {"thing":"this bar that"}
# Out: {"has_foo":false}

escape_html

Escapes HTML special characters.

Examples

root.escaped = this.value.escape_html()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo &amp; bar"}
root.safe_html = this.user_input.escape_html()

# In:  {"user_input":"<script>alert('xss')</script>"}
# Out: {"safe_html":"&lt;script&gt;alert(&#39;xss&#39;)&lt;/script&gt;"}

escape_url_path

Escapes a string for use in URL paths.

Examples

root.escaped = this.value.escape_url_path()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo%20&%20bar"}
root.url = "https://example.com/docs/" + this.path.escape_url_path()

# In:  {"path":"my document.pdf"}
# Out: {"url":"https://example.com/docs/my%20document.pdf"}

escape_url_query

Escapes a string for use in URL query parameters.

Examples

root.escaped = this.value.escape_url_query()

# In:  {"value":"foo & bar"}
# Out: {"escaped":"foo+%26+bar"}
root.url = "https://example.com?search=" + this.query.escape_url_query()

# In:  {"query":"hello world!"}
# Out: {"url":"https://example.com?search=hello+world%21"}

filepath_join

Joins filepath components into a single path.

Examples

root.path = this.path_elements.filepath_join()

# In:  {"path_elements":["/foo/","bar.txt"]}
# Out: {"path":"/foo/bar.txt"}

filepath_split

Splits a filepath into directory and filename components.

Examples

root.path_sep = this.path.filepath_split()

# In:  {"path":"/foo/bar.txt"}
# Out: {"path_sep":["/foo/","bar.txt"]}

# In:  {"path":"baz.txt"}
# Out: {"path_sep":["","baz.txt"]}

format

Formats a value using a specified format string.

Examples

root.foo = "%s(%v): %v".format(this.name, this.age, this.fingers)

# In:  {"name":"lance","age":37,"fingers":13}
# Out: {"foo":"lance(37): 13"}
root.message = "User %s has %v points".format(this.username, this.score)

# In:  {"username":"alice","score":100}
# Out: {"message":"User alice has 100 points"}

has_prefix

Tests if a string starts with a specified prefix.

Parameters

Name Type Description

value

string

The string to test.

Examples

root.t1 = this.v1.has_prefix("foo")
root.t2 = this.v2.has_prefix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":true,"t2":false}

has_suffix

Tests if a string ends with a specified suffix.

Parameters

Name Type Description

value

string

The string to test.

Examples

root.t1 = this.v1.has_suffix("foo")
root.t2 = this.v2.has_suffix("foo")

# In:  {"v1":"foobar","v2":"barfoo"}
# Out: {"t1":false,"t2":true}

index_of

Returns the index of the first occurrence of a substring.

Parameters

Name Type Description

value

string

A string to search for.

Examples

root.index = this.thing.index_of("bar")

# In:  {"thing":"foobar"}
# Out: {"index":3}
root.index = content().index_of("meow")

# In:  the cat meowed, the dog woofed
# Out: {"index":8}

length

Returns the length of an array, object, or string.

Examples

root.foo_len = this.foo.length()

# In:  {"foo":"hello world"}
# Out: {"foo_len":11}
root.foo_len = this.foo.length()

# In:  {"foo":["first","second"]}
# Out: {"foo_len":2}

# In:  {"foo":{"first":"bar","second":"baz"}}
# Out: {"foo_len":2}

lowercase

Converts all letters in a string to lowercase.

Examples

root.foo = this.foo.lowercase()

# In:  {"foo":"HELLO WORLD"}
# Out: {"foo":"hello world"}
root.email = this.user_email.lowercase()

# In:  {"user_email":"User@Example.COM"}
# Out: {"email":"user@example.com"}

quote

Wraps a string in double quotes and escapes special characters.

Examples

root.quoted = this.thing.quote()

# In:  {"thing":"foo\nbar"}
# Out: {"quoted":"\"foo\\nbar\""}
root.literal = this.text.quote()

# In:  {"text":"hello\tworld"}
# Out: {"literal":"\"hello\\tworld\""}

repeat

Creates a string by repeating the input a specified number of times.

Parameters

Name Type Description

count

integer

The number of times to repeat the string.

Examples

root.repeated = this.name.repeat(3)
root.not_repeated = this.name.repeat(0)

# In:  {"name":"bob"}
# Out: {"not_repeated":"","repeated":"bobbobbob"}
root.separator = "-".repeat(10)

# In:  {}
# Out: {"separator":"----------"}

replace

Replaces all occurrences of a substring with another string. Use for text transformation, cleaning data, or normalizing strings.

Parameters

Name Type Description

old

string

A string to match against.

new

string

A string to replace with.

replace_all

Replaces all occurrences of a substring with another.

Parameters

Name Type Description

old

string

A string to match against.

new

string

A string to replace with.

Examples

root.new_value = this.value.replace_all("foo","dog")

# In:  {"value":"The foo ate my homework"}
# Out: {"new_value":"The dog ate my homework"}
root.clean = this.text.replace_all("  ", " ")

# In:  {"text":"hello  world  foo"}
# Out: {"clean":"hello world foo"}

replace_all_many

Performs multiple find-and-replace operations in sequence.

Parameters

Name Type Description

values

array

An array of values, each even value will be replaced with the following odd value.

Examples

root.new_value = this.value.replace_all_many([
  "<b>", "&lt;b&gt;",
  "</b>", "&lt;/b&gt;",
  "<i>", "&lt;i&gt;",
  "</i>", "&lt;/i&gt;",
])

# In:  {"value":"<i>Hello</i> <b>World</b>"}
# Out: {"new_value":"&lt;i&gt;Hello&lt;/i&gt; &lt;b&gt;World&lt;/b&gt;"}

replace_many

Performs multiple find-and-replace operations in sequence using an array of [old, new] pairs. More efficient than chaining multiple replace_all calls. Use for bulk text transformations.

Parameters

Name Type Description

values

array

An array of values, each even value will be replaced with the following odd value.

reverse

Reverses the order of characters in a string.

Examples

root.reversed = this.thing.reverse()

# In:  {"thing":"backwards"}
# Out: {"reversed":"sdrawkcab"}
root = content().reverse()

# In:  {"thing":"backwards"}
# Out: }"sdrawkcab":"gniht"{

slice

Extracts a portion of an array or string.

Parameters

Name Type Description

low

integer

The low bound, which is the first element of the selection, or if negative selects from the end.

high (optional)

integer

An optional high bound.

Examples

root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":"foo bar"}
# Out: {"beginning":"fo","end":"bar"}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-4)
root.the_rest = this.value.slice(0, -4)

# In:  {"value":"foo bar"}
# Out: {"last_chunk":" bar","the_rest":"foo"}
root.beginning = this.value.slice(0, 2)
root.end = this.value.slice(4)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"beginning":["foo","bar"],"end":["bev"]}

A negative low index can be used, indicating an offset from the end of the sequence. If the low index is greater than the length of the sequence then an empty result is returned:

root.last_chunk = this.value.slice(-2)
root.the_rest = this.value.slice(0, -2)

# In:  {"value":["foo","bar","baz","buz","bev"]}
# Out: {"last_chunk":["buz","bev"],"the_rest":["foo","bar","baz"]}

slug

Converts a string into a URL-friendly slug by replacing spaces with hyphens, removing special characters, and converting to lowercase. Supports multiple languages for proper transliteration of non-ASCII characters.

Parameters

Name Type Description

lang (optional)

string

Examples

Create a URL-friendly slug from a string with special characters:

root.slug = this.title.slug()

# In:  {"title":"Hello World! Welcome to Redpanda Connect"}
# Out: {"slug":"hello-world-welcome-to-redpanda-connect"}

Create a slug preserving French language rules:

root.slug = this.title.slug("fr")

# In:  {"title":"Café & Restaurant"}
# Out: {"slug":"cafe-et-restaurant"}

split

Splits a string into an array of substrings.

Parameters

Name Type Description

delimiter

string

The delimiter to split with.

empty_as_null

bool

To treat empty substrings as null values

Examples

root.new_value = this.value.split(",")

# In:  {"value":"foo,bar,baz"}
# Out: {"new_value":["foo","bar","baz"]}
root.new_value = this.value.split(",", true)

# In:  {"value":"foo,,qux"}
# Out: {"new_value":["foo",null,"qux"]}
root.words = this.sentence.split(" ")

# In:  {"sentence":"hello world from bloblang"}
# Out: {"words":["hello","world","from","bloblang"]}

strip_html

Removes HTML tags from a string, returning only the text content. Useful for extracting plain text from HTML documents, sanitizing user input, or preparing content for text analysis. Optionally preserves specific HTML elements while stripping all others.

Parameters

Name Type Description

preserve (optional)

unknown

Optional array of HTML element names to preserve (e.g., ["strong", "em", "a"]). All other HTML tags will be removed.

Examples

Extract plain text from HTML content:

root.plain_text = this.html_content.strip_html()

# In:  {"html_content":"<p>Welcome to <strong>Redpanda Connect</strong>!</p>"}
# Out: {"plain_text":"Welcome to Redpanda Connect!"}

Preserve specific HTML elements while removing others:

root.sanitized = this.html.strip_html(["strong", "em"])

# In:  {"html":"<div><p>Some <strong>bold</strong> and <em>italic</em> text with a <script>alert('xss')</script></p></div>"}
# Out: {"sanitized":"Some <strong>bold</strong> and <em>italic</em> text with a "}

trim

Removes leading and trailing characters from a string.

Parameters

Name Type Description

cutset (optional)

string

An optional string of characters to trim from the target value.

Examples

root.title = this.title.trim("!?")
root.description = this.description.trim()

# In:  {"description":"  something happened and its amazing! ","title":"!!!watch out!?"}
# Out: {"description":"something happened and its amazing!","title":"watch out"}

trim_prefix

Removes a specified prefix from the beginning of a string.

Parameters

Name Type Description

prefix

string

The leading prefix substring to trim from the string.

Examples

root.name = this.name.trim_prefix("foobar_")
root.description = this.description.trim_prefix("foobar_")

# In:  {"description":"unchanged","name":"foobar_blobton"}
# Out: {"description":"unchanged","name":"blobton"}

trim_suffix

Removes a specified suffix from the end of a string.

Parameters

Name Type Description

suffix

string

The trailing suffix substring to trim from the string.

Examples

root.name = this.name.trim_suffix("_foobar")
root.description = this.description.trim_suffix("_foobar")

# In:  {"description":"unchanged","name":"blobton_foobar"}
# Out: {"description":"unchanged","name":"blobton"}

unescape_html

Converts HTML entities back to their original characters.

Examples

root.unescaped = this.value.unescape_html()

# In:  {"value":"foo &amp; bar"}
# Out: {"unescaped":"foo & bar"}
root.text = this.html.unescape_html()

# In:  {"html":"&lt;p&gt;Hello &amp; goodbye&lt;/p&gt;"}
# Out: {"text":"<p>Hello & goodbye</p>"}

unescape_url_path

Unescapes URL path encoding.

Examples

root.unescaped = this.value.unescape_url_path()

# In:  {"value":"foo%20&%20bar"}
# Out: {"unescaped":"foo & bar"}
root.filename = this.path.unescape_url_path()

# In:  {"path":"my%20document.pdf"}
# Out: {"filename":"my document.pdf"}

unescape_url_query

Unescapes URL query parameter encoding.

Examples

root.unescaped = this.value.unescape_url_query()

# In:  {"value":"foo+%26+bar"}
# Out: {"unescaped":"foo & bar"}
root.search = this.param.unescape_url_query()

# In:  {"param":"hello+world%21"}
# Out: {"search":"hello world!"}

unicode_segments

Splits text into segments based on Unicode text segmentation rules. Returns an array of strings representing individual graphemes (visual characters), words (including punctuation and whitespace), or sentences. Handles complex Unicode correctly, including emoji with skin tone modifiers and zero-width joiners.

Parameters

Name Type Description

segmentation_type

string

Type of segmentation: "grapheme", "word", or "sentence"

Examples

Split text into sentences (preserves trailing spaces):

root.sentences = this.text.unicode_segments("sentence")

# In:  {"text":"Hello world. How are you?"}
# Out: {"sentences":["Hello world. ","How are you?"]}

Split text into grapheme clusters (handles complex emoji correctly):

root.graphemes = this.emoji.unicode_segments("grapheme")

# In:  {"emoji":"👨‍👩‍👧‍👦❤️"}
# Out: {"graphemes":["👨‍👩‍👧‍👦","❤️"]}

unquote

Removes surrounding quotes and interprets escape sequences.

Examples

root.unquoted = this.thing.unquote()

# In:  {"thing":"\"foo\\nbar\""}
# Out: {"unquoted":"foo\nbar"}
root.text = this.literal.unquote()

# In:  {"literal":"\"hello\\tworld\""}
# Out: {"text":"hello\tworld"}

uppercase

Converts all letters in a string to uppercase.

Examples

root.foo = this.foo.uppercase()

# In:  {"foo":"hello world"}
# Out: {"foo":"HELLO WORLD"}
root.code = this.product_code.uppercase()

# In:  {"product_code":"abc-123"}
# Out: {"code":"ABC-123"}

Timestamp manipulation

parse_duration

Parses a Go-style duration string into nanoseconds. A duration string is a signed sequence of decimal numbers with unit suffixes like "300ms", "-1.5h", or "2h45m". Valid units: "ns", "us" (or "µs"), "ms", "s", "m", "h".

Examples

Parse microseconds to nanoseconds:

root.delay_for_ns = this.delay_for.parse_duration()

# In:  {"delay_for":"50us"}
# Out: {"delay_for_ns":50000}

Parse hours to seconds:

root.delay_for_s = this.delay_for.parse_duration() / 1000000000

# In:  {"delay_for":"2h"}
# Out: {"delay_for_s":7200}

parse_duration_iso8601

Parses an ISO 8601 duration string into nanoseconds. Format: "P[n]Y[n]M[n]DT[n]H[n]M[n]S" or "P[n]W". Example: "P3Y6M4DT12H30M5S" means 3 years, 6 months, 4 days, 12 hours, 30 minutes, 5 seconds. Supports fractional seconds with full precision (not just one decimal place).

Examples

Parse complex ISO 8601 duration to nanoseconds:

root.delay_for_ns = this.delay_for.parse_duration_iso8601()

# In:  {"delay_for":"P3Y6M4DT12H30M5S"}
# Out: {"delay_for_ns":110839937000000000}

Parse hours to seconds:

root.delay_for_s = this.delay_for.parse_duration_iso8601() / 1000000000

# In:  {"delay_for":"PT2H"}
# Out: {"delay_for_s":7200}

ts_add_iso8601

Adds an ISO 8601 duration to a timestamp with calendar-aware precision for years, months, and days. Useful when you need to add durations that account for variable month lengths or leap years.

Parameters

Name Type Description

duration

string

Duration in ISO 8601 format (e.g., "P1Y2M3D" for 1 year, 2 months, 3 days)

Examples

Add one year to a timestamp:

root.next_year = this.created_at.ts_add_iso8601("P1Y")

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"next_year":"2021-08-14T05:54:23Z"}

Add a complex duration with multiple units:

root.future_date = this.created_at.ts_add_iso8601("P1Y2M3DT4H5M6S")

# In:  {"created_at":"2020-01-01T00:00:00Z"}
# Out: {"future_date":"2021-03-04T04:05:06Z"}

ts_format

Formats a timestamp as a string using Go’s reference time format. Defaults to RFC 3339 if no format specified. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference. Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Use ts_strftime for strftime-style formats.

Parameters

Name Type Description

format

string

The output format using Go’s reference time.

tz (optional)

string

Optional timezone (e.g., 'UTC', 'America/New_York'). Defaults to input timezone or local time for unix timestamps.

Examples

Format timestamp with custom format:

root.something_at = this.created_at.ts_format("2006-Jan-02 15:04:05")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}

Format unix timestamp with timezone specification:

root.something_at = this.created_at.ts_format(format: "2006-Jan-02 15:04:05", tz: "UTC")

# In:  {"created_at":1597405526}
# Out: {"something_at":"2020-Aug-14 11:45:26"}

ts_parse

Parses a timestamp string using Go’s reference time format and outputs a timestamp object. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference - show how this reference time would appear in your format. Use ts_strptime for strftime-style formats instead.

Parameters

Name Type Description

format

string

The format of the input string using Go’s reference time.

Examples

Parse a date with abbreviated month name:

root.doc.timestamp = this.doc.timestamp.ts_parse("2006-Jan-02")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}

Parse a custom datetime format:

root.parsed = this.timestamp.ts_parse("Jan 2, 2006 at 3:04pm (MST)")

# In:  {"timestamp":"Aug 14, 2020 at 5:54am (UTC)"}
# Out: {"parsed":"2020-08-14T05:54:00Z"}

ts_round

Rounds a timestamp to the nearest multiple of the specified duration. Halfway values round up. Accepts unix timestamps (seconds with optional decimal precision) or RFC 3339 formatted strings.

Parameters

Name Type Description

duration

integer

A duration measured in nanoseconds to round by.

Examples

Round timestamp to the nearest hour:

root.created_at_hour = this.created_at.ts_round("1h".parse_duration())

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"created_at_hour":"2020-08-14T06:00:00Z"}

Round timestamp to the nearest minute:

root.created_at_minute = this.created_at.ts_round("1m".parse_duration())

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"created_at_minute":"2020-08-14T05:54:00Z"}

ts_strftime

Formats a timestamp as a string using strptime format specifiers (like %Y, %m, %d). Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Supports %f for microseconds. Use ts_format for Go-style reference time formats.

Parameters

Name Type Description

format

string

The output format using strptime specifiers.

tz (optional)

string

Optional timezone. Defaults to input timezone or local time for unix timestamps.

Examples

Format timestamp with strftime specifiers:

root.something_at = this.created_at.ts_strftime("%Y-%b-%d %H:%M:%S")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26"}

Format with microseconds using %f directive:

root.something_at = this.created_at.ts_strftime("%Y-%b-%d %H:%M:%S.%f", "UTC")

# In:  {"created_at":"2020-08-14T11:50:26.371Z"}
# Out: {"something_at":"2020-Aug-14 11:50:26.371000"}

ts_strptime

Parses a timestamp string using strptime format specifiers (like %Y, %m, %d) and outputs a timestamp object. Use ts_parse for Go-style reference time formats instead.

Parameters

Name Type Description

format

string

The format string using strptime specifiers (e.g., %Y-%m-%d).

Examples

Parse date with abbreviated month using strptime format:

root.doc.timestamp = this.doc.timestamp.ts_strptime("%Y-%b-%d")

# In:  {"doc":{"timestamp":"2020-Aug-14"}}
# Out: {"doc":{"timestamp":"2020-08-14T00:00:00Z"}}

Parse datetime with microseconds using %f directive:

root.doc.timestamp = this.doc.timestamp.ts_strptime("%Y-%b-%d %H:%M:%S.%f")

# In:  {"doc":{"timestamp":"2020-Aug-14 11:50:26.371000"}}
# Out: {"doc":{"timestamp":"2020-08-14T11:50:26.371Z"}}

ts_sub

Calculates the duration in nanoseconds between two timestamps (t1 - t2). Returns a signed integer: positive if t1 is after t2, negative if t1 is before t2. Use .abs() for absolute duration.

Parameters

Name Type Description

t2

timestamp

The timestamp to subtract from the target timestamp.

Examples

Calculate absolute duration between two timestamps:

root.between = this.started_at.ts_sub("2020-08-14T05:54:23Z").abs()

# In:  {"started_at":"2020-08-13T05:54:23Z"}
# Out: {"between":86400000000000}

Calculate signed duration (can be negative):

root.duration_ns = this.end_time.ts_sub(this.start_time)

# In:  {"start_time":"2020-08-14T10:00:00Z","end_time":"2020-08-14T11:30:00Z"}
# Out: {"duration_ns":5400000000000}

ts_sub_iso8601

Subtracts an ISO 8601 duration from a timestamp with calendar-aware precision for years, months, and days. Useful when you need to subtract durations that account for variable month lengths or leap years.

Parameters

Name Type Description

duration

string

Duration in ISO 8601 format (e.g., "P1Y2M3D" for 1 year, 2 months, 3 days)

Examples

Subtract one year from a timestamp:

root.last_year = this.created_at.ts_sub_iso8601("P1Y")

# In:  {"created_at":"2020-08-14T05:54:23Z"}
# Out: {"last_year":"2019-08-14T05:54:23Z"}

Subtract a complex duration with multiple units:

root.past_date = this.created_at.ts_sub_iso8601("P1Y2M3DT4H5M6S")

# In:  {"created_at":"2021-03-04T04:05:06Z"}
# Out: {"past_date":"2020-01-01T00:00:00Z"}

ts_tz

Converts a timestamp to a different timezone while preserving the moment in time. Accepts unix timestamps (seconds with optional decimal precision) or RFC 3339 formatted strings.

Parameters

Name Type Description

tz

string

The timezone to change to. Use "UTC" for UTC, "Local" for local timezone, or an IANA Time Zone database location name like "America/New_York".

Examples

Convert timestamp to UTC timezone:

root.created_at_utc = this.created_at.ts_tz("UTC")

# In:  {"created_at":"2021-02-03T17:05:06+01:00"}
# Out: {"created_at_utc":"2021-02-03T16:05:06Z"}

Convert timestamp to a specific timezone:

root.created_at_ny = this.created_at.ts_tz("America/New_York")

# In:  {"created_at":"2021-02-03T16:05:06Z"}
# Out: {"created_at_ny":"2021-02-03T11:05:06-05:00"}

ts_unix

Converts a timestamp to a unix timestamp (seconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing seconds.

Examples

Convert RFC 3339 timestamp to unix seconds:

root.created_at_unix = this.created_at.ts_unix()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000}

Unix timestamp passthrough returns same value:

root.timestamp = this.ts.ts_unix()

# In:  {"ts":1257894000}
# Out: {"timestamp":1257894000}

ts_unix_micro

Converts a timestamp to a unix timestamp with microsecond precision (microseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing microseconds.

Examples

Convert timestamp to microseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_micro()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000000}

Preserve microsecond precision from timestamp:

root.precise_time = this.timestamp.ts_unix_micro()

# In:  {"timestamp":"2020-08-14T11:45:26.123456Z"}
# Out: {"precise_time":1597405526123456}

ts_unix_milli

Converts a timestamp to a unix timestamp with millisecond precision (milliseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing milliseconds.

Examples

Convert timestamp to milliseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_milli()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000}

Useful for JavaScript timestamp compatibility:

root.js_timestamp = this.event_time.ts_unix_milli()

# In:  {"event_time":"2020-08-14T11:45:26.123Z"}
# Out: {"js_timestamp":1597405526123}

ts_unix_nano

Converts a timestamp to a unix timestamp with nanosecond precision (nanoseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing nanoseconds.

Examples

Convert timestamp to nanoseconds since epoch:

root.created_at_unix = this.created_at.ts_unix_nano()

# In:  {"created_at":"2009-11-10T23:00:00Z"}
# Out: {"created_at_unix":1257894000000000000}

Preserve full nanosecond precision:

root.precise_time = this.timestamp.ts_unix_nano()

# In:  {"timestamp":"2020-08-14T11:45:26.123456789Z"}
# Out: {"precise_time":1597405526123456789}

Type coercion

array

Converts a value to an array.

Examples

root.my_array = this.name.array()

# In:  {"name":"foobar bazson"}
# Out: {"my_array":["foobar bazson"]}

bool

Converts a value to a boolean with optional fallback.

Parameters

Name Type Description

default (optional)

bool

An optional value to yield if the target cannot be parsed as a boolean.

Examples

root.foo = this.thing.bool()
root.bar = this.thing.bool(true)

bytes

Marshals a value into a byte array.

Examples

root.first_byte = this.name.bytes().index(0)

# In:  {"name":"foobar bazson"}
# Out: {"first_byte":102}

not_empty

Ensures a value is not empty.

Examples

root.a = this.a.not_empty()

# In:  {"a":"foo"}
# Out: {"a":"foo"}

# In:  {"a":""}
# Out: Error("failed assignment (line 1): field `this.a`: string value is empty")

# In:  {"a":["foo","bar"]}
# Out: {"a":["foo","bar"]}

# In:  {"a":[]}
# Out: Error("failed assignment (line 1): field `this.a`: array value is empty")

# In:  {"a":{"b":"foo","c":"bar"}}
# Out: {"a":{"b":"foo","c":"bar"}}

# In:  {"a":{}}
# Out: Error("failed assignment (line 1): field `this.a`: object value is empty")

not_null

Ensures a value is not null.

Examples

root.a = this.a.not_null()

# In:  {"a":"foobar","b":"barbaz"}
# Out: {"a":"foobar"}

# In:  {"b":"barbaz"}
# Out: Error("failed assignment (line 1): field `this.a`: value is null")

number

Converts a value to a number with optional fallback.

Parameters

Name Type Description

default (optional)

float

An optional value to yield if the target cannot be parsed as a number.

Examples

root.foo = this.thing.number() + 10
root.bar = this.thing.number(5) * 10

string

Converts a value to a string representation.

Examples

root.nested_json = this.string()

# In:  {"foo":"bar"}
# Out: {"nested_json":"{\"foo\":\"bar\"}"}
root.id = this.id.string()

# In:  {"id":228930314431312345}
# Out: {"id":"228930314431312345"}

timestamp

Converts a value to a timestamp with optional fallback.

Parameters

Name Type Description

default (optional)

timestamp

An optional value to yield if the target cannot be parsed as a timestamp.

Examples

root.foo = this.ts.timestamp()
root.bar = this.none.timestamp(1234567890.timestamp())

type

Returns the type of a value as a string.

Examples

root.bar_type = this.bar.type()
root.foo_type = this.foo.type()

# In:  {"bar":10,"foo":"is a string"}
# Out: {"bar_type":"number","foo_type":"string"}
root.type = this.type()

# In:  "foobar"
# Out: {"type":"string"}

# In:  666
# Out: {"type":"number"}

# In:  false
# Out: {"type":"bool"}

# In:  ["foo", "bar"]
# Out: {"type":"array"}

# In:  {"foo": "bar"}
# Out: {"type":"object"}

# In:  null
# Out: {"type":"null"}
root.type = content().type()

# In:  foobar
# Out: {"type":"bytes"}
root.type = this.ts_parse("2006-01-02").type()

# In:  "2022-06-06"
# Out: {"type":"timestamp"}

Deprecated

format_timestamp

This method is deprecated and will be removed in a future version.

Formats a timestamp as a string using Go’s reference time format. Defaults to RFC 3339 if no format specified. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference. Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Use ts_strftime for strftime-style formats.

Parameters

Name Type Description

format

string

The output format using Go’s reference time.

tz (optional)

string

Optional timezone (e.g., 'UTC', 'America/New_York'). Defaults to input timezone or local time for unix timestamps.

format_timestamp_strftime

This method is deprecated and will be removed in a future version.

Formats a timestamp as a string using strptime format specifiers (like %Y, %m, %d). Accepts unix timestamps (with decimal precision) or RFC 3339 strings. Supports %f for microseconds. Use ts_format for Go-style reference time formats.

Parameters

Name Type Description

format

string

The output format using strptime specifiers.

tz (optional)

string

Optional timezone. Defaults to input timezone or local time for unix timestamps.

format_timestamp_unix

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp (seconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing seconds.

format_timestamp_unix_micro

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with microsecond precision (microseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing microseconds.

format_timestamp_unix_milli

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with millisecond precision (milliseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing milliseconds.

format_timestamp_unix_nano

This method is deprecated and will be removed in a future version.

Converts a timestamp to a unix timestamp with nanosecond precision (nanoseconds since epoch). Accepts unix timestamps or RFC 3339 strings. Returns an integer representing nanoseconds.

parse_timestamp

This method is deprecated and will be removed in a future version.

Parses a timestamp string using Go’s reference time format and outputs a timestamp object. The format uses "Mon Jan 2 15:04:05 -0700 MST 2006" as a reference - show how this reference time would appear in your format. Use ts_strptime for strftime-style formats instead.

Parameters

Name Type Description

format

string

The format of the input string using Go’s reference time.

parse_timestamp_strptime

This method is deprecated and will be removed in a future version.

Parses a timestamp string using strptime format specifiers (like %Y, %m, %d) and outputs a timestamp object. Use ts_parse for Go-style reference time formats instead.

Parameters

Name Type Description

format

string

The format string using strptime specifiers (e.g., %Y-%m-%d).