EZAudio.png

EZAudio

   

An iOS and OSX audio visualization framework built upon Core Audio useful for anyone doing real-time, low-latency audio processing and visualizations.

HIGHLIGHTS

Circular Audio Waveform

Segmented Audio Waveform

Rehearse (Pitch and Time Shifter)

EZAudio is a popular open source iOS/macOS audio framework I wrote that's used to perform a variety of common audio related tasks on iPhone, iPad, and macOS. I wrote EZAudio initially as a playground where I could quickly utilize common audio components to function at a high-level, while still having access to the audio data. For example, getting audio input from any device's microphone in 1 line of code, reading compressed and uncompressed audio files a consistent way, or creating really good looking waveforms.

EZAudio has since been used around the world in tons of different apps and is currently the most starred iOS/macOS audio framework on Github.


COMPONENTS

The following components are available in EZAudio and most can be chained together to create an audio signal flow. In addition, most of the audio classes also provide the audio data as an array of floats to allow seamless use of the audio waveform components.

EZAudioDevice

Selecting an input audio device on iOS using EZAudioDevice

The EZAudioDevice serves as a utility class for getting the current device's available inputs and outputs. This class is especially useful for selecting the appropriate input and output sources for the EZMicrophone and EZOutput, respectively.

iOS

Inputs include the bottom, front, and back microphones. Outputs include the external speakers and headphone jack.

macOS

For macOS this would be any of the input devices available in the System Preferences/Sound window. Similarly, in addition to the default speakers and Airplay, any of the output sources in the System Preferences/Sound window can be used as an output device. For instance, a digital-to-audio device used for getting instrument input and external monitoring like the Focusrite Scarlett 2i2 could be used for both input and output from/to EZAudio.

Getting audio input on iOS using EZMicrophone

The EZMicrophone serves as the main class for providing audio input data. The input data is provided in two formats, one as an AudioBufferList data structure for use in recording and playback, and the other as an array of floats (each representing one channel of audio data) that can be used in the EZAudioPlot for plotting the audio waveform. Additionally, an EZAudioDevice can be set to pull audio data from a specific input source.

EZRecorder

Recording audio input on iOS using EZRecorder

The EZRecorder serves as the main class for writing audio data to a file. This class supports writing the audio data in various compressed and uncompressed formats including m4a, aiff, and wav. The current time, duration, and number of frames are dynamic properties that allow developers to properly show UI elements related to the recorder's state. 

EZAudioFile

Audio File 2.png

The EZAudioFile serves as the main class for reading audio data for playback and generating waveform data for audio plots. This class is used by the EZAudioPlayer to perform audio playback (internally the player uses the EZOutput).

EZOutput

The EZOutput serves as the main class for sending audio data to the device's speakers. It can take in any type of audio data (including compressed formats) and will internally convert it to a linear PCM format. In addition to modifying the volume and pan of the output, the EZOutput class can be subclassed to allow extending the signal chain with additional Audio Unit effects (inspired by Daniel Kennett's blog post on modifying the audio signal chain in CocoaLibSpotify). An EZAudioDevice can be set to specify which output source the audio should play out of.

EZAudioPlayer

Player.png

Playing an audio file on iOS using the EZAudioPlayer (internally using the EZAudioFile and EZOutput)

The EZAudioPlayer is a utility class used for performing audio file playback (internally using the EZAudioFile and EZOutput). The player provides notifications for play state changes, frame-precise seeking, volume control, stereo panning, as well as output device switching.

EZAudioPlot

The EZAudioPlot displaying microphone input on iOS and switching between the buffer and rolling styles.

The EZAudioPlot is a UI class used to display audio waveforms using Core Graphics and Core Animation. The plot is capable of updating at 60 fps with minimal CPU usage since the compositing step of the CAShapeLayer allows us to update the underlying bezier path using the GPU. The plot supports displaying waveforms in either the Buffer (realtime values) or Rolling (trailing history of values) styles.

EZAudioPlotGL

The EZAudioPlotGL displaying microphone input on iOS and switching between the buffer and rolling styles.

The EZAudioPlotGL displaying microphone input on iOS and switching between the buffer and rolling styles.

The EZAudioPlotGL is a UI class used to display audio waveforms using OpenGL. The plot is optimized for updating at 60 fps (or higher) with minimal CPU usage since all work is done on the GPU. The plot supports displaying waveforms in either the Buffer (realtime values) or Rolling (trailing history of values) styles. Users can also subclass the EZAudioPlotGL to perform their own custom waveform drawing using the audio data (check out the Circular and Segmented audio plots above).

EZAudioFFT

A pitch detector example on iOS using the EZAudioFFT.

The EZAudioFFT is a utility class used for calculating realtime Fast Fourier Transforms (FFTs) using the Accelerate framework. There is also an EZAudioFFTRolling class that calculates higher accuracy FFTs (smaller bins in the frequency domain) by maintaining a trailing history buffer of values. 

EZAudioFloatConverter

The EZAudioFloatConverter is a utility class used for converting audio from any input format to a stereo, non-interleaved linear PCM format. Though this class is used mostly internally by other classes such as the EZMicrophone and EZOutput for providing audio data as an array of floats for audio waveform plotting, but it can be used directly for doing format conversion


Deprecated

On June 13, 2016 I deprecated EZAudio to give myself time to focus on my work at Skycatch. I still use EZAudio in my own personal projects and have recently developed a streaming component that allows time and pitch shifting, but for now I can't officially say I'm back to maintaining it full-time.

As of now I highly recommend using AudioKit for developing iOS/macOS/tvOS audio-related apps!