Review Article Vol. 5, No.

12 / December 2018 / Optica 1623

Linear programmable nanophotonic processors

NICHOLAS C. HARRIS,1 JACQUES CAROLAN,2 DARIUS BUNANDAR,2 MIHIKA PRABHU,2 MICHAEL HOCHBERG,3
TOM BAEHR-JONES,3 MICHAEL L. FANTO,4 A. MATTHEW SMITH,4 CHRISTOPHER C. TISON,4
PAUL M. ALSING,4 AND DIRK ENGLUND2,*
1
Lightmatter, 61 Chatham St 5th floor, Boston, Massachusetts 02109, USA
2
Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, Massachusetts 02139, USA
3
Elenion Technologies, New York, New York 10016, USA
4
Air Force Research Laboratory, Information Directorate, 525 Brooks Road, Rome, New York 13441, USA
*Corresponding author: [email protected]

Received 22 August 2018; revised 16 October 2018; accepted 25 October 2018 (Doc. ID 341321); published 20 December 2018

Advances in photonic integrated circuits have recently enabled electrically reconfigurable optical systems that can
implement universal linear optics transformations on spatial mode sets. This review paper covers progress in such
“programmable nanophotonic processors” as well as emerging applications of the technology to problems including
classical and quantum information processing and machine learning. © 2018 Optical Society of America under the terms of
the OSA Open Access Publishing Agreement

https://doi.org/10.1364/OPTICA.5.001623

1. INTRODUCTION photonic components are imperfect, nearly perfect mode trans-

Photonic integrated circuits (PICs) have become increasingly formations become possible in sufficiently large reconfigurable
important in classical communications applications over the past optical devices [13,14]—indicating that scaling optical systems
decades, including as transmitters and receivers in long-haul, while mitigating errors is feasible. Among reconfigurable optical
metro, and data center interconnects. Many of the attributes that systems, there has been much progress towards “universal linear
make PICs attractive for these applications—compactness, high optics” devices: photonic circuits that can be programmed to per-
bandwidth, and the ability to control large numbers of optical form all possible linear optical transformations on a given set
modes with high phase stability—also make them appealing of input modes [15,16]. This paper will review progress towards
for entirely new applications, such as hardware accelerators such general-purpose “programmable nanophotonic processors”
based on emerging classical and quantum computing concepts. (PNPs) and emerging applications to problems including
However, these emerging applications come with highly demand- machine learning and quantum information processing. The
ing device and scaling requirements. For example, proposed op- PNPs considered here implement linear optical transformations
tical matrix processors will likely require the control of at least by one-way propagation; we assume no resonators or other feed-
hundreds of spatial modes to be useful as neural network hard- back loops, which are important for a number of applications,
ware accelerators [1–3], optical quantum computing protocols including RF filtering [17–20].
may require similar numbers of optical modes for each logical
quantum bit (qubit) [4–6], and quantum computing schemes 2. PROGRAMMABLE NANOPHOTONIC
based on atomic memories will also require high-performance PROCESSORS
control over large numbers of optical spatial modes [7–9]. In ad-
dition, many of these emerging applications will require new types The most popular methods for constructing a programmable
of devices, such as extremely low-loss modulators, and may need mode transformer from N input to N output modes break the
to function at wavelengths outside the standard telecommunica- problem up into a mesh of 2 × 2 mode transformers consisting of
tions band. Whereas these challenges may have appeared daunt- Mach–Zehnder interferometers (MZIs) [13,15,21], as shown in
ing a decade or two ago, rapid advances in PICs have recently Figs. 1(a) and 1(b). Each MZI consists of two 50% beam
enabled proof-of-concept demonstrations. Silicon-on-insulator splitters and two phase shifters parameterized by
θ, ϕ, as shown
(SOI), silicon nitride, and indium phosphide (InP) technology to the right of Fig. 1(b). In integrated photonics platforms,
has, in many areas, led the way thanks in large part to the avail- beam splitters are commonly realized by directional couplers
ability of mature fabrication processes and multi-project-wafer that convert input modes a1 , a2 into output modes b1,2 
(MPW) services [10–12]. Recently, SOI PIC systems that can p1ffiffi
a 1,2  ia2,1 ; note the π∕2 phase in the cross terms guarantees
2
coherently control tens of optical modes have been demonstrated unitarity of the directional coupler transformation. The MZI
[1]. Crucially, it was shown that even though all individual shown in the inset of Fig. 1(b) applies the SU
2 transformation,

2334-2536/18/121623-09 Journal © 2018 Optical Society of America

 Review Article Vol. 5, No. 12 / December 2018 / Optica 1624

 
e iϕ sin
θ∕2 e iϕ cos
θ∕2 and 1(b). A cascaded binary tree structure [23] that can imple-
U
2  ,
cos
θ∕2 − sin
θ∕2 ment arbitrary unitary transformations has also been proposed.
The network shown in Fig. 1(c) was originally proposed by
up to a global phase. Here we assume the unit cell is lossless; ac- Miller as a method for realizing any linear transformation on a
counting for losses requires each MZI to be described by a 4 × 4 set of spatial modes [16]. This network uses a physical instantia-
matrix, rather than the 2 × 2 matrix considered here. Losses can tion of the singular value decomposition, which is a factorization
be modeled by “virtual” beam splitters coupling the original mode of any matrix (M ) as M  U ΣV † , where U is an m × m unitary
and a “vacuum” mode. If such virtual beam splitters are included, matrix; Σ is an m × n diagonal, rectangular matrix of nonnegative
the overall transformation can still be represented as a unitary real numbers; and V is an n × n unitary matrix. Here, two uni-
U
M , where M > N accounts for the additional loss channels. versal unitary circuits (U , V † ) are connected by a column of sin-
The N × N transformation that applies to our input and output gle MZIs that are used as variable attenuators implementing Σ.
waveguide modes then comprise a nonunitary submatrix of In the original implementation of the “Miller” network, each
U
M . In instances where the loss in each component is iden- MZI is implemented using two internal phase shifters with the
tical, it is possible to represent the PNP transformation by the differential phase between the two phase shifters being one
unitary U
N  and to account for loss as a global parameter α ≤ parameter and the global phase imparted by the two phase shifters
1 that can be factored out as V
N   αU
N . Experimentally, as another parameter [13,16,23–25]. The “Miller” MZI configu-
waveguide losses have been shown to be relatively uniform, so that ration can be more compact than the standard configuration,
losses can likely be assumed to be uniformly distributed [10]. In since the overall unit cell length is reduced by the length of one
this case, the scattering statistics for q identical, single photons phase shifter.
passing through the PNP are described by U
N , and the prob- PNPs have been demonstrated in a number of material plat-
abilities of all photons arriving at the output will scale as αq . forms, some of which are summarized in Fig. 2. The SOI platform
For a universal unitary transformation, each of the N input offers an especially high index contrast of 3.4:1.5, which enables
modes must be coupled to each of the N output modes. low-loss waveguide bends with radii as small as 2 μm [28]. The
Figure 1(a) shows an arrangement of MZIs connecting N  6 resulting high component densities are especially important for
modes. To allow connections between all modes, one requires large PNPs, which already can have up to 88 MZIs connecting
Σn  N
N − 1∕2 (N choose 2) MZIs—15 MZIs for this exam- 26 optical [1] modes, as shown in Fig. 2(a), and applications are
ple. The triangular arrangement of Fig. 1(a) was first proposed by demanding much larger devices. Figure 2(b) shows a silicon pho-
Reck et al. [22]. Figure 1(b) shows a more compact arrangement, tonics-based U
4 PNP that was used for separating a multimode
described by Clements et al. [21], that accomplishes the same channel into individual single-mode waveguides. The U
6 PNP
U
N  transformation; it also requires 15 MZIs for N  6 was realized in germanium-doped glass with thermal modulators,
modes. Both the “Reck” and “Clements” decomposition algo- illustrated in Fig. 2(c), and enabled the demonstration of
rithms terminate with a matrix that implements U
N  up to a linear optical quantum gates and boson sampling schemes [15].
diagonal phase screen. The phase screen can be implemented Figure 2(d) shows a silicon photonics-based U
4 PNP used to
using phase shifters at each input mode, as shown in Figs. 1(a) demonstrate a universal coupler [26].

(a) (b)

(c)

Fig. 1. (a), (b) Universal unitary networks composed of MZIs; (a) shows the “Reck” encoding and (b) shows the “Clements” encoding. Inset shows the
unit cell of a PNP, a programmable MZI. (c) Universal linear network composed of two universal unitary circuits and an additional column of “loss” MZIs
originally described by Miller [16].
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1625

Fig. 2. (a) Optical micrograph of 26-mode, 88-MZI PNP [1]. PCBs are visible above and below the chip. (b) Artistic rendering of a U
4 PNP by
Annoni et al. [26]. (c) Germanium-doped glass six-mode, 15-MZI PNP by Carolan et al. [15]. (d) Four-mode, six-MZI PNP by Ribeiro et al. [27]
implemented in the SOI platform.

Phase shifter technology in MZIs is of central importance, phase between the two input modes to an MZI can be controlled
and a number of phase shifter technologies are being advanced. and is described by some phase γ. Without an external phase
Lossless phase shifting mechanisms in silicon include the thermo- shifter (ϕ), transformations are confined to the plane shown in
optic effect (3 dB bandwidth up to a few hundred kilohertz) [29], Fig. 3(b). To access the full Poincaré sphere, an external phase
mechanical effects (∼MHz bandwidth) [30,31], and electric- shifter is required.
field-induced electro-optic effects (∼GHz bandwidth) [32]. A number of programming protocols have been developed,
Recent work [33] has investigated the integration of III-V mate- and they can broadly be grouped into one of three categories:
rials with silicon photonics for compact, low-power phase shifting (1) element-by-element, with phase shifter settings for each
based on metal-oxide semiconductor capacitors. The possibility of MZI considered individually; (2) compiled, with phase shifter set-
monolithically integrated silicon transistor control circuits [34] tings for each MZI resulting from a matrix decomposition algo-
and photonic components bolsters the case for large-scale PNPs rithm [16,21,22]; or (3) optimized, with phase shifter settings for
in silicon. Phase modulation mechanisms that introduce dynamic each MZI resulting from the execution of an optimization pro-
loss, such as the plasma dispersion effect, are not ideal for realizing tocol acting on the phase shifters [1,16].
PNPs since they complicate the description of the MZI unit cell PNPs acting as a switching matrix are generally programmed
and give rise to nonunitary transformations. A number of avenues using a category (1) protocol. PNPs implementing matrices or
exist to further increase component density. One example is to quantum gates [2,15] (which can be specified as unitary matrices)
shrink the directional couplers. Inverse design methods are par- are generally programmed using a category (2) protocol. A matrix
ticularly promising for producing wavelength-scale devices [35,36]. is provided as input to a decomposition algorithm, which then
returns the phase shifter settings required to realize the matrix
transformation. PNPs used as black boxes that unscramble light
3. PNP PROGRAMMING [26] or scatter light to implement a specific output intensity pat-
Configuring or programming N × N mode transformations in a tern [1] are programmed using a category (3) protocol where the
PNP involves precise tuning of approximately N 2 phases. This phase shifter settings are prescribed by an optimization algorithm.
can be a nontrivial problem, especially when considering MZI To evaluate the accuracy of a program in a PNP, it is useful
inhomogeneity and the potential for cross talk between modula- to characterize the unitary transformation it implements.
tors (especially relevant for thermal modulators). MZI phases are Fortunately, efficient techniques exist [37] that use laser light
set by applying voltages or currents to each phase shifter, labeled to determine the amplitude elements (jU i,j j2 ) and interferometry
here as
i, j within the array. Figure 3(c) outlines the basic pro- to determine the phase arguments (arg
U i,j ) up to an unobserv-
gramming flow. Before considering possible routes towards pro- able input and output phase screen. Circuit fidelity is a metric that
gramming an entire PNP, it is instructive to consider the behavior quantifies the “closeness” between two unitary matrices and is
of a single, programmable MZI. Some single-MZI programming given by F C  Tr
jU † U T j2 , where Tr
 is the trace operator,
examples are shown in Table 1; here, we assume the differential U is the measured unitary, and U T is a target unitary.
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1626

(a) for this initial phase disorder, a PNP can be calibrated. There
are several known algorithms for calibration, including self-
configuring protocols [23] and progressive algorithms [39]. The
ability to monitor the power at each MZI in a PNP enables dy-
namic, local measurements of the state of the system (at the cost
of electronic control complexity); contactless integrated photonic
probe (CLIPP) detectors avoid excess insertion loss by detecting
light via bandgap defect states [26].

4. APPLICATIONS
(b) We now discuss a subset of recent PNP applications: self-
configuration and mode mixing, quantum transport and quan-
tum gates, and machine learning.
A. Self-Configuration
As mentioned above, accurate configuration of the many degrees
of freedom (phase settings) in the PNP can pose a challenge,
especially when accounting for inhomogeneity in constituent de-
vices. In 2013, Miller proposed a self-configuring solution for one
particular PNP function: the coherent addition of light from N
(c)
spatial input modes into one spatial output mode by canceling the
fields in the remaining N − 1 output modes [16]. This concept is
illustrated in Figs. 4(a) and 4(b), where the phase shifters of MZIs
A–D are consecutively tuned to cancel the photocurrents on the
corresponding output detectors. An important advantage in this
approach is that each MZI can function without global knowl-
edge of the other MZIs or photodetectors, and this independent
self-configuration promises that such coherent, nearly lossless
mode adders could be very fast. The coherent field adder only
works if the optical modes are locally phase stable; for example,
Fig. 3. (a) Phase shifter addressing scheme. (b) Poincaré sphere show-
it would be impossible to add single-photon excitations (which
ing the space of transformations enabled between the top “t” and bottom
“b” waveguide modes. Without an external phase shifter, transformations
have no fixed relative phase) over the input modes. Instead,
are confined to the blue plane; with an external phase shifter, transfor- arbitrary linear optical mode converters require an N × N mesh.
mations span the sphere. (c) Programming model for programmable Using an extension of his previous work, Miller proposed such a
nanophotonic processors. After each round of programming, the results self-configuring N × N mesh that uses detectors on each MZI [26].
of the measurement step can be used to correct the program. Using SOI PIC platforms, a 4 × 4 universal PNP with power mon-
itoring taps was demonstrated in 2016 [27] [see Fig. 2(d)]. A 4 × 4
dynamically self-configuring mode adder was demonstrated in 2017
[26]. As shown in Fig. 4(c), the authors used a 980 nm laser to
Table 1. Example Matrices That Can Be Generated by a generate a dynamic input state to the 4 × 4 mesh and used CLIPP
Single MZI with an Internal Phase Shifter, an External Phase detectors to actively track and undo mode mixing. As they scale in
Shifter ϕ, and Control over the Input Phase Difference γa numbers of modes, self-configuring circuits could enable a range of
applications [40], from spatial multiplexing/demultiplexing—for
Gate Matrix (γ, θ, ϕ)
  example in multimode fiber communications—to beam tracking
1 1
Hadamard p1ffiffi
2 1 −1
(0, π∕2, 0) and quantum circuits. The “Clements” architecture cannot be self-
  configured in this way, though a scheme has been proposed to allow
0 1
Pauli-X (σ̂ x ) (0, 0, 0) progressive configuration of such networks [24].
1 0
 
0 −i B. Quantum Information Processing
Pauli-Y (σ̂ y ) (π∕2, 0, π∕2)
i 0
  Photons are appealing as a carrier of quantum information due
1 0
Pauli-Z (σ̂ z ) (0, π, 0) to their ability to propagate with low loss over long distances,
0 −1
a
By setting θ  π∕2 and all other phases to zero, the Hadamard matrix (or
phase stability, and their amenability to control even at room
50:50 splitter) is realized. temperature in PICs [41]. Perhaps the greatest challenge lies
in producing controlled interactions between photonic quantum
states: deterministic two-photon gates require many ancillae
After fabrication, the initial state of the PNP is unknown due photons together with measurement and fast active feedforward
to static phase disorder within the waveguides. This effect has [9,42,43], or atom-mediated interactions [44,45] translated to
been studied in the context of silicon photonics and is parame- PIC-compatible platforms [46–49]. As both approaches require
terized by the static “phase coherence length” [38]; in silicon, this phase-stable control of large numbers of optical modes with
parameter is typically on the order of a few millimeters. To correct high precision, programmable PNPs are emerging as important
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1627

(a)
(b)

(c) (c.i) (c.ii) (c.iii)

Fig. 4. (a) Schematic of system for coherent summing of light from N input spatial modes. (b) Schematic drawing of a U
4, Reck-topology PNP with
a four-input, four-output multimode interferometer tied to the input waveguides. Active MZIs in this experiment are highlighted blue. (c) To implement
dynamic mode mixing, 980 nm light is focused on the multimode interferometer. Eye diagram for signal passing through the mixer (c.i) without the
perturbing laser, (c.ii) with the perturbing laser and automatic calibration disabled, and (c.iii) with the perturbing laser and automatic calibration enabled.

platforms. In contrast to custom-built static PICs, PNPs also chains of coupled quantum systems. One experimental approach
provide a platform for rapid prototyping of photonic quantum relies on a photonic quantum walk along discrete lattice sites,
information processing protocols, including quantum comput- which can be represented as the waveguides of the PNP. While
ing protocols [15], quantum transport [1], and quantum simu- nonlinear interactions between photons give rise to particularly
lation [6,50]. In the following, we briefly discuss some of these rich phenomena and applications, even linear quantum walks
demonstrations. of single or multiple photons have a number of applications
[1,50,55,56] and have been proposed to be computationally hard
1. Quantum Transport on classical computers for large-enough problems [57].
A number of interesting problems, from coherent effects in bio- Figure 5(a) shows the topology of a 26-mode, fan-out PNP
logical processes [51] to quantum computing [52,53] and quan- implemented in the silicon photonic platform; this PNP consists
tum search [54], involve the transport of quantum particles along of 88 programmable MZIs and 176 phase shifters and supports

(a) (c)

(d)

(b)

Fig. 5. (a) Schematic representation of the 26-mode PNP along with the coordinate system definition for quantum transport experiments.
(b) Conceptual drawing of the phase landscape for a strong, statically disordered system where light is localized initially to waveguide i 14 . By introducing
dynamic phase disorder (shown as red vibrations), it is possible to optimize transport of light to distant waveguide sites.
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1628

embeddings of universal unitary circuits up to U
9. An input of heralded gates in both the circuit and cluster state model.
state of photons enters from the left and undergoes a quantum Figures 6(a) and 6(b) show the symbol and photonic circuit
walk on a 1D chain as it passes in time along the right. By for a heralded controlled-NOT (CNOT) operation, which uses
programming the splitting ratios of the sites (via the internal phase two path-encoded computational photons and two ancillary pho-
shift), it is possible to explore discrete-site quantum transport tons. Given a detection in the ancillary modes, the CNOT logic
on a number of graphs. In a recent experiment, Harris et al. [1] is guaranteed to have taken place on the computational photons
explored a single photonic quantum walker under static and dy- [see Fig. 6(c)]. Technologically, the low coupling loss of 0.4 dB
namic phase disorder. Each of the MZIs were set to implement between silica waveguides and input/output fibers was key to en-
50:50 splitting ratios, but the external phase shifters were pro- abling multiphoton experiments of up to six photons. While SOI
grammed to have either a static phase variation [illustrated in PNPs have so far been limited to coupling losses of 3 dB, losses as
Fig. 5(c)], a dynamically changing phase [illustrated in Fig. 5(d)], low as 0.4 dB have been demonstrated in silicon photonics [61],
or any combination of static and dynamic phase variations. In this pointing the way towards large-scale SOI PNPs suitable for multi-
configuration, the PNP implements a balanced coin quantum photon quantum information.
walk on a discrete-time, 1D graph. A sufficiently large static-only
phase variation can confine photons to a local vicinity (as in C. Machine Learning
Anderson localization), whereas a strong dynamic phase variation Artificial neural network (ANN) algorithms have dramatically im-
causes a ballistic diffusion in time (due to dephasing between the proved natural language processing, image recognition, object
sites). An optimal trade-off between static and dynamic disorder detection, and more [62]. ANNs rely heavily on matrix-vector
(which rises with effective system temperature) had been pre- products and require frequent memory access during training and
dicted to facilitate environment-assisted exciton transport in pho- inference. Recent work has focused on developing tailored elec-
tosynthetic complexes [51]. In this regime, dynamic disorder tronics architecture for ANNs that take advantage of the limited
prevents a particle from becoming “stuck” in one site. The pro- requirements on computational precision, large matrix sparsity,
grammability of the PNP made it possible to carefully study this and other features to achieve improved computational rates and
quantum transport across 64,400 unique settings of static and energy efficiency [63–68]. However, the computational speed and
dynamic disorder, and demonstrate this environment-assisted power efficiency achieved with these hardware architectures are
quantum transport experimentally. still bound by underlying transistor device physics, including
switching energies and electronic clock rates—two quantities
2. Quantum Gates that are closely linked.
Universal quantum computers follow two predominant frame- Some machine learning algorithms, including neural net-
works: the circuit model [58], where single qubit and multiqubit works, appear suited for analog computing architectures, includ-
gates are performed sequentially on qubits, and the cluster state ing analog complementary metal-oxide semiconductor (CMOS)
model [59,60], where a large entangled resource state is first cre- circuits [69], memristor arrays [70,71], photonic networks [2],
ated, and then single qubit gates are performed, which encode the and magnetic devices [72]. Photonic methods may simultane-
computation. In linear optics photonic quantum computing, ously enable low latency, high energy efficiency, and high
two-qubit processes are realized probabilistically. It is therefore throughput [2]. While bulk-optical implementations of optical
critical that the successful operation of a gate be “heralded” by neural networks (ONNs) have been suggested in the past [73],
ancillary photons. Carolan et al. [15] used a six-mode PNP it has only recently become possible to implement large-scale,
alongside an off-chip multiphoton source to implement a variety phase-stable, and programmable linear transformations. Recent
work has focused on implementing hybrid optical-electronic
systems that implement spike processing [74] and reservoir com-
puting [75–77]. Augmented with optical nonlinearities, PNPs
promise high-speed and low-power implementations of neural
networks fully in the optical domain.
As shown by Shen and Harris et al. [2], it is possible to directly
map the mathematical description of a multilayer perceptron,
the most basic form of deep neural network, onto arrays of
PNPs connected by nonlinear optical components. In each layer
of a multilayer perceptron, a matrix-vector product is evaluated,
and then each entry of the resultant output vector is passed
through a nonlinear “activation function.” A schematic repre-
sentation of an ONN is shown in Fig. 7(a), and a zoom into
a single layer is shown in Fig. 7(b). Matrix-vector products are
evaluated using optical interference units in the “Miller” encoding
[(PNPs implementing arbitrary, nonunitary matrices as shown
in Fig. 1(c)] [16], and activation functions are realized with an
Fig. 6. Linear optical quantum logic gates in a PNP [15]. (a) Heralded optical nonlinearity unit (ONU). Vectors are encoded in the
controlled-NOT gate schematic. (b) Program within the U
6 PNP. intensity and phase distribution of optical signals incident at
(c) Computational truth table, with theoretical result overlaid. the left of the ONN. These optical signals propagate through
Correspondence between MZI reflectivities and colored beam splitters the set of layers comprising the ONN and are finally converted
in (b) shown at right. into electrical current using detectors, shown at the right of
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1629

(a) systems with over 1000 active elements and the circuits that con-
trol them have been monolithically integrated in CMOS proc-
esses [86]; MEMS and NEMS switches show promise for low-
power switch arrays [27,31]; and a growing range of materials
are becoming available, including SOI, silicon nitride, and
InP. These developments point to a new era in photonics design
(b) and applications in which high-volume manufacturing will make
general purpose PNPs containing an abundance of components
cost-effective over custom-designed PICs in many applications.
As field programmable gate arrays (FPGAs) have enabled a
new paradigm for electronics, PNPs, or “optical FPGAs,” will en-
able unforeseen applications and advances for optical processing.
Fig. 7. (a) Optical neural network architecture overview. (b) Zoom
into a single layer of the neural network. The optical interference unit Funding. Air Force Office of Scientific Research (AFOSR)
can be realized, as shown in Fig. 1(b). (FA8750-14-2-0120, FA9550-13-1-0027, FA9550-14-1-0052);
Air Force Research Laboratory (AFRL) Program (FA8750-14-
2-0120, FA8750-16-2-0141); Office of the Secretary of Defense
Fig. 7(a). An ONU could be implemented using saturable absorb- (OSD); Applied Research for Advanced Science and Technology
ers [78,79] or devices that exhibit bistability [80–82]; both kinds (ARAP) Quantum Science and Engineering Program (QSEP)
of nonlinear optical devices have been demonstrated in integrated program.
photonic systems, but challenges remain in realizing an array of
such nonlinear devices in a single system. Acknowledgment. D. E. would like to acknowledge sup-
Existing neural network training algorithms, such as backpro- port from AFRL. M. L. F., A. M. S., C. C. T., and P. M. A.
pagation [83,84], executed on electronic computers can be used would like to acknowledge support of this work from OSD.
to determine the set of matrices to be programmed into the Any opinions, findings, conclusions, or recommendations ex-
ONN. After training, a set of weights in each layer that minimizes pressed in this material are those of the author(s) and do not nec-
an error metric is determined. These weight matrices can be de- essarily reflect the views of Air Force Research Laboratory.
composed into PNP phase shifter settings at each layer. After pro-
gramming, the ONN can be used as an inference machine— REFERENCES
classifying vectors that are not part of the training data set. 1. N. C. Harris, G. R. Steinbrecher, M. Prabhu, Y. Lahini, J. Mower, D.
This adaptation of deep neural networks to integrated photon- Bunandar, C. Chen, F. N. C. Wong, T. Baehr-Jones, M. Hochberg, S.
ics was tested on a simple vowel recognition problem [2]. A two- Lloyd, and D. Englund, “Quantum transport simulations in a program-
layer neural network with four neurons per layer and a saturable mable nanophotonic processor,” Nat. Photonics 11, 447–452 (2017).
2. Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M.
absorber nonlinear activation function was trained on a 64-bit
Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M.
computer against a set of four-dimensional input vectors that re- Soljačić, “Deep learning with coherent nanophotonic circuits,” Nat.
present recordings of people speaking one of four vowels. The Photonics 11, 441–446 (2017).
data set contained 360 vectors; 180 were used for training and 3. A. N. Tait, T. F. de Lima, E. Zhou, A. X. Wu, M. A. Nahmias, B. J. Shastri,
180 were exclusively used for testing. After training, the ONN and P. R. Prucnal, “Neuromorphic photonic networks using silicon pho-
tonic weight banks,” Sci. Rep. 7, 7430 (2017).
was able to correctly classify 138/180 spoken vowels (compared 4. M. Pant, H. Krovi, D. Englund, and S. Guha, “Rate-distance tradeoff and
to 165/180 for a 64-bit digital computer). Advances in PNP pro- resource costs for all-optical quantum repeaters,” Phys. Rev. A 95,
gramming fidelity and improved readout (including optical fiber 012304 (2017).
packaging techniques) may reduce the performance gap between 5. K. Kieling, T. Rudolph, and J. Eisert, “Percolation, renormalization, and
quantum computing with nondeterministic gates,” Phys. Rev. Lett. 99,
the ONN and the digitally simulated one.
130501 (2007).
6. A. Aspuru-Guzik and P. Walther, “Photonic quantum simulators,” Nat.
Phys. 8, 285–291 (2012).
5. DISCUSSION 7. Y. Li, P. C. Humphreys, G. J. Mendoza, and S. C. Benjamin, “Resource
PNPs are already finding applications in proof-of-concept dem- costs for fault-tolerant linear optical quantum computing,” Phys. Rev. X 5,
onstrations including classical computing systems [1–3], quan- 041007 (2015).
8. K. Nemoto, M. Trupke, S. J. Devitt, A. M. Stephens, B. Scharfenberger,
tum computing systems [15], self-calibrating mode mixers K. Buczak, T. Nöbauer, M. S. Everitt, J. Schmiedmayer, and W. J. Munro,
[26], and matrix processors [2,15,27]. For real-world applica- “Photonic architecture for scalable quantum information processing in
tions, it is still necessary to address some important challenges, diamond,” Phys. Rev. X 4, 031022 (2014).
including (1) the development of more compact, low-power 9. M. Pant, D. Towsley, D. Englund, and S. Guha, “Percolation thresholds
for photonic quantum computing,” arXiv:1701.03775 (2017).
phase shifters with ultralow loss and—for many applications— 10. T. Baehr-Jones, R. Ding, A. Ayazi, T. Pinguet, M. Streshinsky, N. Harris,
programmability at rates of MHz and higher; (2) operation out- J. Li, L. He, M. Gould, Y. Zhang, A. Eu-Jin Lim, T.-Y. Liow, S. Hwee-Gee
side the near-infrared spectrum, especially at shorter wavelengths; Teo, G.-Q. Lo, and M. Hochberg, “A 25 Gb/s silicon photonics platform,”
(3) precise electronic control over tens of thousands of phase shift- arXiv:1203.0767 (2012).
ers; and (4) more compact ultralow loss passive components, 11. P. P. Absil, P. De Heyn, H. Chen, P. Verheyen, G. Lepage, M.
Pantouvaki, J. De Coster, A. Khanna, Y. Drissi, D. Van Thourhout,
which may be developed by computational design [35,85]. and J. Van Campenhout, “Imec iSiPP25G silicon photonics: a robust
While there are many challenges towards scaling PNPs, signifi- CMOS-based photonics technology platform,” Proc. SPIE 9367,
cant progress is being made on multiple fronts. Optoelectronic 93670V (2015).
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1630

12. K. Wörhoff, R. G. Heideman, A. Leinse, and M. Hoekman, “TriPleX: a 37. S. Rahimi-Keshari, M. A. Broome, R. Fickler, A. Fedrizzi, T. C. Ralph,
versatile dielectric photonic platform,” Adv. Opt. Technol. 4, 189–207 and A. G. White, “Direct characterization of linear-optical networks,”
(2015). Opt. Express 21, 13450–13458 (2013).
13. D. A. B. Miller, “Perfect optics with imperfect components,” Optica 2, 38. Y. Yang, Y. Ma, H. Guan, Y. Liu, S. Danziger, S. Ocheltree, K. Bergman,
747–750 (2015). T. Baehr-Jones, and M. Hochberg, “Phase coherence length in silicon
14. J. Mower, N. C. Harris, G. R. Steinbrecher, Y. Lahini, and D. Englund, photonic platform,” Opt. Express 23, 16890–16902 (2015).
“High-fidelity quantum state evolution in imperfect photonic integrated 39. N. C. Harris, “Programmable nanophotonics for quantum information
circuits,” Phys. Rev. A 92, 032322 (2015). processing and artificial intelligence,” Ph.D. thesis (Massachusetts
15. J. Carolan, C. Harrold, C. Sparrow, E. Martín-López, N. J. Russell, J. W. Institute of Technology, 2017).
Silverstone, P. J. Shadbolt, N. Matsuda, M. Oguma, M. Itoh, G. D. 40. D. A. B. Miller, “Sorting out light,” Science 347, 1423–1424 (2015).
Marshall, M. G. Thompson, J. C. F. Matthews, T. Hashimoto, J. L. 41. J. W. Silverstone, D. Bonneau, J. L. O’Brien, and M. G. Thompson,
O’Brien, and A. Laing, “Universal linear optics,” Science 349, 711– “Silicon quantum photonics,” IEEE J. Sel. Top. Quantum Electron. 22,
716 (2015). 390–402 (2016).
16. D. A. B. Miller, “Self-configuring universal linear optical component (in- 42. P. Kok, W. J. Munro, K. Nemoto, T. C. Ralph, J. P. Dowling, and G. J.
vited),” Photon. Res. 1, 1–15 (2013). Milburn, “Linear optical quantum computing with photonic qubits,” Rev.
17. D. Pérez, I. Gasulla, and J. Capmany, “Programmable multifunctional Mod. Phys. 79, 135–174 (2007).
integrated nanophotonics,” Nanophotonics 7, 1351–1371 (2018). 43. T. Rudolph, “Why I am optimistic about the silicon-photonic route to
18. D. Pérez, I. Gasulla, L. Crudgington, D. J. Thomson, A. Z. Khokhar, K. Li, quantum computing,” APL Photon. 2, 030901 (2017).
W. Cao, G. Z. Mashanovich, and J. Capmany, “Multipurpose silicon pho- 44. B. Hacker, S. Welte, G. Rempe, and S. Ritter, “A photon–photon quan-
tonics signal processor core,” Nat. Commun. 8, 636 (2017). tum gate based on a single atom in an optical resonator,” Nature 536,
19. D. Pérez, I. Gasulla, J. Capmany, and R. A. Soref, “Reconfigurable lattice 193–196 (2016).
mesh designs for programmable photonic processors,” Opt. Express 24, 45. D. J. Brod and J. Combes, “Passive CPHASE gate via cross-Kerr non-
12093–12106 (2016). linearities,” Phys. Rev. Lett. 117, 080502 (2016).
20. L. Zhuang, C. G. Roeloffzen, M. Hoekman, K.-J. Boller, and A. J. Lowery, 46. S. Sun, H. Kim, Z. Luo, G. S. Solomon, and E. Waks, “A single-photon
“Programmable photonic signal processor chip for radiofrequency appli- switch and transistor enabled by a solid-state quantum memory,”
cations,” Optica 2, 854–859 (2015). Science 361, 57–60 (2018).
21. W. R. Clements, P. C. Humphreys, B. J. Metcalf, W. Steven Kolthammer, 47. A. Sipahigil, R. E. Evans, D. D. Sukachev, M. J. Burek, J. Borregaard, M.
and I. A. Walmsley, “Optimal design for universal multiport interferome- K. Bhaskar, C. T. Nguyen, J. L. Pacheco, H. A. Atikian, C. Meuwly, R. M.
ters,” Optica 3, 1460–1465 (2016). Camacho, F. Jelezko, E. Bielejec, H. Park, M. Lončar, and M. D. Lukin,
22. M. Reck, A. Zeilinger, H. J. Bernstein, and P. Bertani, “Experimental “An integrated diamond nanophotonics platform for quantum optical net-
realization of any discrete unitary operator,” Phys. Rev. Lett. 73, works,” Science 354, 847–850 (2016).
58–61 (1994). 48. J.-H. Kim, S. Aghaeimeibodi, C. J. K. Richardson, R. P. Leavitt, D.
23. D. A. B. Miller, “Self-aligning universal beam coupler,” Opt. Express 21, Englund, and E. Waks, “Hybrid integration of solid-state quantum
6360–6370 (2013). emitters on a silicon photonic chip,” Nano Lett. 17, 7394–7400 (2017).
24. D. A. B. Miller, “Setting up meshes of interferometers—reversed local 49. S. L. Mouradian, T. Schröder, C. B. Poitras, L. Li, J. Goldstein, E. H.
light interference method,” Opt. Express 25, 29233–29248 (2017). Chen, M. Walsh, J. Cardenas, M. L. Markham, D. J. Twitchen, M.
25. D. A. Miller, “How complicated must an optical component be?” J. Opt. Lipson, and D. Englund, “Scalable integration of long-lived quantum
Soc. Am. A 30, 238–251 (2013). memories into a photonic circuit,” Phys. Rev. X 5, 031009 (2015).
26. A. Annoni, E. Guglielmi, M. Carminati, G. Ferrari, M. Sampietro, D. A. B. 50. C. Sparrow, E. Martn-López, N. Maraviglia, A. Neville, C. Harrold, J.
Miller, A. Melloni, and F. Morichetti, “Unscrambling light—automatically Carolan, Y. N. Joglekar, T. Hashimoto, N. Matsuda, J. L. O’Brien, D.
undoing strong mixing between modes,” Light Sci. Appl. 6, e17110 (2017). P. Tew, and A. Laing, “Simulating the vibrational quantum dynamics
27. A. Ribeiro, A. Ruocco, L. Vanacker, and W. Bogaerts, “Demonstration of of molecules using photonics,” Nature 557, 660–667 (2018).
a 4 × 4-port universal linear circuit,” Optica 3, 1348–1357 (2016). 51. P. Rebentrost, M. Mohseni, I. Kassal, S. Lloyd, and A. Aspuru-Guzik,
28. M. R. Watts, “Adiabatic microring resonators,” Opt. Lett. 35, 3231–3233 “Environment-assisted quantum transport,” New J. Phys. 11, 033003
(2010). (2009).
29. N. C. Harris, Y. Ma, J. Mower, T. Baehr-Jones, D. Englund, M. Hochberg, 52. A. M. Childs, D. Gosset, and Z. Webb, “Universal computation by multi-
and C. Galland, “Efficient, compact and low loss thermo-optic phase particle quantum walk,” Science 339, 791–794 (2013).
shifter in silicon,” Opt. Express 22, 10487–10493 (2014). 53. Y. Lahini, G. R. Steinbrecher, A. D. Bookatz, and D. Englund, “Quantum
30. M. Poot and H. X. Tang, “Broadband nanoelectromechanical phase shift- logic using correlated one-dimensional quantum walks,” npj Quantum
ing of light on a chip,” Appl. Phys. Lett. 104, 061101 (2014). Inf. 4, 2 (2018).
31. S. Han, T. J. Seok, N. Quack, B.-W. Yoo, and M. C. Wu, “Large-scale 54. S. Aaronson and A. Ambainis, “Quantum search of spatial regions,” in
silicon photonic switches with movable directional couplers,” Optica 2, 44th Annual IEEE Symposium on Foundations of Computer Science
370–375 (2015). (2003), pp. 200–209.
32. E. Timurdogan, C. V. Poulton, M. J. Byrd, and M. R. Watts, “Electric field- 55. A. Peruzzo, M. Lobino, J. C. F. Matthews, N. Matsuda, A. Politi, K.
induced second-order nonlinear optical effects in silicon waveguides,” Poulios, X.-Q. Zhou, Y. Lahini, N. Ismail, K. Wörhoff, Y. Bromberg, Y.
Nat. Photonics 11, 200–206 (2017). Silberberg, M. G. Thompson, and J. L. O’Brien, “Quantum walks of cor-
33. M. Takenaka, J.-H. Han, J.-K. Park, F. Boeuf, J. Fujikata, S. Takahashi, related photons,” Science 329, 1500–1503 (2010).
and S. Takagi, “High-efficiency, low-loss optical phase modulator based 56. A. Crespi, R. Osellame, R. Ramponi, V. Giovannetti, R. Fazio, L.
on III-V/Si hybrid MOS capacitor,” in Optical Fiber Communication Sansoni, F. De Nicola, F. Sciarrino, and P. Mataloni, “Anderson locali-
Conference (Optical Society of America, 2018), paper Tu3K.3. zation of entangled photons in an integrated quantum walk,” Nat.
34. C. Sun, M. T. Wade, Y. Lee, J. S. Orcutt, L. Alloatti, M. S. Georgas, A. S. Photonics 7, 322–328 (2013).
Waterman, J. M. Shainline, R. R. Avizienis, S. Lin, B. R. Moss, R. Kumar, 57. S. Aaronson and A. Arkhipov, “The computational complexity of linear
F. Pavanello, A. H. Atabaki, H. M. Cook, A. J. Ou, J. C. Leu, Y.-H. Chen, optics,” in 43rd Annual ACM Symposium on Theory of Computing
K. Asanović, R. J. Ram, M. Popović, and V. M. Stojanović, “Single-chip (STOC) (ACM, 2011), pp. 333–342.
microprocessor that communicates directly using light,” Nature 528, 58. M. A. Nielsen and I. Chuang, “Quantum computation and quantum infor-
534–538 (2015). mation,” Am. J. Phys. 70, 558–559 (2002).
35. A. Y. Piggott, J. Lu, K. G. Lagoudakis, J. Petykiewicz, T. M. Babinec, and 59. R. Raussendorf and H. J. Briegel, “A one-way quantum computer,” Phys.
J. Vučković, “Inverse design and demonstration of a compact and broad- Rev. Lett. 86, 5188–5191 (2001).
band on-chip wavelength demultiplexer,” Nat. Photonics 9, 374–377 60. R. Raussendorf, D. E. Browne, and H. J. Briegel, “Measurement-based
(2015). quantum computation on cluster states,” Phys. Rev. A 68, 022312
36. Y. Zhang, S. Yang, A. E.-J. Lim, G.-Q. Lo, C. Galland, T. Baehr-Jones, (2003).
and M. Hochberg, “A compact and low loss Y-junction for submicron 61. J. Notaros, F. Pavanello, M. T. Wade, C. M. Gentry, A. Atabaki, L.
silicon waveguide,” Opt. Express 21, 1310–1316 (2013). Alloatti, R. J. Ram, and M. A. Popović, “Ultra-efficient CMOS fiber-to-chip
 Review Article Vol. 5, No. 12 / December 2018 / Optica 1631

grating couplers,” in Optical Fiber Communications Conference and 73. N. H. Farhat, D. Psaltis, A. Prata, and E. Paek, “Optical implementation
Exhibition (OFC) (2016), pp. 1–3. of the Hopfield model,” Appl. Opt. 24, 1469–1475 (1985).
62. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436– 74. S. K. Esser, P. A. Merolla, J. V. Arthur, A. S. Cassidy, R. Appuswamy, A.
444 (2015). Andreopoulos, D. J. Berg, J. L. McKinstry, T. Melano, D. R. Barch, C. di
63. C.-S. Poon and K. Zhou, “Neuromorphic silicon neurons and large-scale Nolfo, P. Datta, A. Amir, B. Taba, M. D. Flickner, and D. S. Modha,
neural networks: challenges and opportunities,” Front. Neurosci. 5, 108 “Convolutional networks for fast, energy-efficient neuromorphic comput-
(2011). ing,” Proc. Natl. Acad. Sci. USA 113, 11441–11446 (2016).
64. A. Shafiee, A. Nag, N. Muralimanohar, R. Balasubramonian, J. P. 75. K. Vandoorne, P. Mechet, T. Van Vaerenbergh, M. Fiers, G. Morthier,
Strachan, M. Hu, R. S. Williams, and V. Srikumar, “ISAAC: a convolu- D. Verstraeten, B. Schrauwen, J. Dambre, and P. Bienstman,
tional neural network accelerator with in-situ analog arithmetic in cross- “Experimental demonstration of reservoir computing on a silicon photon-
bars,” in ACM/IEEE 43rd Annual International Symposium on Computer ics chip,” Nat. Commun. 5, 3541 (2014).
Architecture (ISCA) (2016), pp. 14–26. 76. L. Larger, M. C. Soriano, D. Brunner, L. Appeltant, J. M. Gutierrez, L.
65. J. Misra and I. Saha, “Artificial neural networks in hardware: a survey of Pesquera, C. R. Mirasso, and I. Fischer, “Photonic information process-
two decades of progress,” Neurocomputing 74, 239–255 (2010). ing beyond Turing: an optoelectronic implementation of reservoir com-
66. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den puting,” Opt. Express 20, 3241–3249 (2012).
Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. 77. Y. Paquot, F. Duport, A. Smerieri, J. Dambre, B. Schrauwen, M.
Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Haelterman, and S. Massar, “Optoelectronic reservoir computing,” Sci.
Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Rep. 2, 287 (2012).
Hassabis, “Mastering the game of Go with deep neural networks and 78. A. C. Selden, “Pulse transmission through a saturable absorber,” Br. J.
tree search,” Nature 529, 484–489 (2016). Appl. Phys. 18, 743–748 (1967).
67. Y. H. Chen, T. Krishna, J. S. Emer, and V. Sze, “Eyeriss: an energy-ef- 79. Q. Bao, H. Zhang, Z. Ni, Y. Wang, L. Polavarapu, Z. Shen, Q.-H. Xu, D.
ficient reconfigurable accelerator for deep convolutional neural net- Tang, and K. P. Loh, “Monolayer graphene as a saturable absorber in a
works,” IEEE J. Solid-State Circuits 52, 127–138 (2017). mode-locked laser,” Nano Res. 4, 297–307 (2011).
68. A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska- 80. B. Xu and N. B. Ming, “Experimental observations of bistability and
Barwińska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, instability in a two-dimensional nonlinear optical superlattice,” Phys.
A. P. Badia, K. M. Hermann, Y. Zwols, G. Ostrovski, A. Cain, H. King, C. Rev. Lett. 71, 3959–3962 (1993).
Summerfield, P. Blunsom, K. Kavukcuoglu, and D. Hassabis, “Hybrid 81. E. Centeno and D. Felbacq, “Optical bistability in finite-size nonlinear bi-
computing using a neural network with dynamic external memory,” dimensional photonic crystals doped by a microcavity,” Phys. Rev. B 62,
Nature 538, 471–476 (2016). R7683–R7686 (2000).
69. A. Biswas and A. P. Chandrakasan, “Conv-RAM: an energy-efficient 82. K. Nozaki, T. Tanabe, A. Shinya, S. Matsuo, T. Sato, H. Taniyama, and
SRAM with embedded convolution computation for low-power CNN- M. Notomi, “Sub-femtojoule all-optical switching using a photonic-crystal
based machine learning applications,” in IEEE International Solid-State nanocavity,” Nat. Photonics 4, 477–483 (2010).
Circuits Conference (ISSCC) (2018), pp. 488–490. 83. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning represen-
70. A. Pantazi, S. Woźniak, T. Tuma, and E. Eleftheriou, “All-memristive neu- tations by back-propagating errors,” Nature 323, 533–536 (1986).
romorphic computing with level-tuned neurons,” Nanotechnology 27, 84. T. W. Hughes, M. Minkov, Y. Shi, and S. Fan, “Training of photonic neural
355205 (2016). networks through in situ backpropagation and gradient measurement,”
71. M. Hu, C. E. Graves, C. Li, Y. Li, N. Ge, E. Montgomery, N. Davila, H. Optica 5, 864–871 (2018).
Jiang, R. S. Williams, J. J. Yang, Q. Xia, and J. P. Strachan, “Memristor- 85. B. Shen, P. Wang, R. Polson, and R. Menon, “An integrated-nanopho-
based analog computation and neural network classification with a dot tonics polarization beamsplitter with 2.4 × 2.4 μm2 footprint,” Nat.
product engine,” Adv. Mater. 30, 1705914 (2018). Photonics 9, 378–382 (2015).
72. J. Torrejon, M. Riou, F. A. Araujo, S. Tsunegi, G. Khalsa, D. Querlioz, P. 86. S. Chung, H. Abediasl, and H. Hashemi, “15.4 A 1024-element
Bortolotti, V. Cros, K. Yakushiji, A. Fukushima, H. Kubota, S. Yuasa, M. scalable optical phased array in 0.18 μm SOI CMOS,” in IEEE
D. Stiles, and J. Grollier, “Neuromorphic computing with nanoscale spin- International Solid-State Circuits Conference (ISSCC) (IEEE, 2017),
tronic oscillators,” Nature 547, 428–431 (2017). pp. 262–263.