TranscodingStreams.jl ===================== [![Docs Stable][docs-stable-img]][docs-stable-url] [![Docs Latest][docs-latest-img]][docs-latest-url] [![Build Status](https://github.com/JuliaIO/TranscodingStreams.jl/actions/workflows/CI.yml/badge.svg?branch=master)](https://github.com/JuliaIO/TranscodingStreams.jl/actions/workflows/CI.yml?query=branch%3Amaster) [![codecov.io][codecov-img]][codecov-url] ![TranscodingStream](/docs/src/assets/transcodingstream.png) TranscodingStreams.jl is a package for transcoding data streams, which is: - **fast**: small overhead and specialized methods, - **consistent**: basic I/O operations you already know will work as you expect, - **generic**: support any I/O objects like files, buffers, pipes, etc., and - **extensible**: you can define a new codec to transcode data. ## Installation ```julia Pkg.add("TranscodingStreams") ``` Installing a [codec package](#codec-packages) will install TranscodingStreams.jl as well, and so in general you don't need to explicitly install it. ## Usage ```julia using TranscodingStreams, CodecZlib # Some text. text = """ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aenean sollicitudin mauris non nisi consectetur, a dapibus urna pretium. Vestibulum non posuere erat. Donec luctus a turpis eget aliquet. Cras tristique iaculis ex, eu malesuada sem interdum sed. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Etiam volutpat, risus nec gravida ultricies, erat ex bibendum ipsum, sed varius ipsum ipsum vitae dui. """ # Streaming API. stream = IOBuffer(text) stream = TranscodingStream(GzipCompressor(), stream) stream = TranscodingStream(GzipDecompressor(), stream) for line in eachline(stream) println(line) end close(stream) # Array API. array = Vector{UInt8}(text) array = transcode(GzipCompressor, array) array = transcode(GzipDecompressor, array) @assert text == String(array) # Pre-allocated API import TranscodingStreams: Buffer compressed_buffer = Buffer(Vector{UInt8}(undef, ncodeunits(text))) decompressed_buffer = Buffer(codeunits(text)) transcode(GzipCompressor, decompressed_buffer, compressed_buffer) transcode(GzipDecompressor, compressed_buffer, decompressed_buffer) @assert text == String(decompressed_buffer.data) ``` Each codec has an alias to its transcoding stream type for ease of use. For example, `GzipCompressorStream{S} = TranscodingStream{GzipCompressor,S} where S<:IO`. Consult the [docs][docs-latest-url] for more details and examples. ## Codec packages TranscodingStreams.jl offers I/O interfaces to users. It also offers a protocol suite to communicate with various codecs. However, specific codecs are not included in this package except the `Noop` codec, which does nothing to data. The user need to install codecs as a plug-in to do something meaningful. The following codec packages support the protocol suite: - [CodecZlib.jl](https://github.com/JuliaIO/CodecZlib.jl) - [CodecXz.jl](https://github.com/JuliaIO/CodecXz.jl) - [CodecZstd.jl](https://github.com/JuliaIO/CodecZstd.jl) - [CodecBase.jl](https://github.com/JuliaIO/CodecBase.jl) - [CodecBzip2.jl](https://github.com/JuliaIO/CodecBzip2.jl) - [CodecLz4.jl](https://github.com/JuliaIO/CodecLz4.jl) by Invenia. [travisci-img]: https://travis-ci.org/JuliaIO/TranscodingStreams.jl.svg?branch=master [travisci-url]: https://travis-ci.org/JuliaIO/TranscodingStreams.jl [codecov-img]: http://codecov.io/github/JuliaIO/TranscodingStreams.jl/coverage.svg?branch=master [codecov-url]: http://codecov.io/github/JuliaIO/TranscodingStreams.jl?branch=master [docs-stable-img]: https://img.shields.io/badge/docs-stable-blue.svg [docs-stable-url]: https://juliaio.github.io/TranscodingStreams.jl/stable/ [docs-latest-img]: https://img.shields.io/badge/docs-latest-blue.svg [docs-latest-url]: https://juliaio.github.io/TranscodingStreams.jl/latest/