Audio Signal Processing in Java and Ptolemy II

Researchers: Brian K. Vogel
Advisor:Edward A. Lee
This project is exploring the implementation of hard-real-time systems, particularly audio signal processing, using abstract models of computation and the Java language. We are currently implementing a particular audio signal processing algorithm [2,3] under various domains in Ptolemy II and as a stand-alone Java application, and evaluating the performance bottlenecks. The algorithm performs formant-corrected pitch shifting of human vocal sounds; it contains a significant amount of control logic as well as stream-based computation, and thus stresses the methods currently available in Ptolemy II.

A short-term objective of the project is to explore the suitability of Java for real-time audio processing. A longer term objective is to evaluate the efficacy of abstracted models of computation such as synchronous dataflow and finite-state machines for such algorithms. Through careful profiling, limitations imposed by the software architecture can be identified and corrected. This will also help us evaluate the long-term potential of virtual-machine architectures for hard-real-time processing.

Finally, since the pitch-shifting algorithm is fairly complex and requires both control flow and data flow, it is a good candidate for exploring several models of computation, including some of the more experimental models such as heterochronous dataflow (HDF) [1]. HDF is more expressive than synchronous dataflow (SDF), yet HDF retains desirable properties such as decidability (bounded memory, deadlock) and static schedules. A potential drawback to HDF is that it may be difficult (nonintuitive) to use. The process of implementing the pitch shifting algorithm in HDF may provide insight as to the intuitiveness of constructing models using this model of computation, and may also influence design decisions such as when to construct static vs. dynamic schedules.

This work is in collaboration with Prof. David Wessel at the Center for New Music and Audio Technologies (CNMAT).

[1]
A. Girault, B. Lee, and E. A. Lee, "Hierarchical Finite State Machines with Multiple Concurrency Models," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, Vol. 18, No. 6, pp.742-760, 1999.
[2]
K. Lent, "An Efficient Method for Pitch Shifting Digitally Sampled Sounds," Computer Music J., vol. 13, pp. 65-71 (1989 Winter).
[3]
R. Bristow-Johnson, "A Detailed Analysis of a Time-Domain Formant-Corrected Pitch-Shifting Algorithm," J. Audio Eng. Soc., vol. 43, no. 5, pp. 340-352 (1995 May).

Last updated 11/05/99