MASSIVELY DISTRIBUTED NEUROMORPHIC CONTROL FOR LEGGED
ROBOTS MODELED AFTER INSECT STEPPING
By
NICHOLAS STEPHEN SZCZECINSKI
Submitted in partial fulfillment of the requirements
For the degree of Master of Science
Thesis Adviser: Dr. Roger D. Quinn
Department of Mechanical Engineering
CASE WESTERN RESERVE UNIVERSITY
January 2013
CASE WESTERN RESERVE UNIVERSITY
SCHOOL OF GRADUATE STUDIES
We hereby approve the thesis of
Nicholas Stephen Szczecinski
Candidate for the Master of Science degree*.
(signed)Roger D. Quinn
(chair of committee)
Roy E. Ritzmann
Michael S. Branicky
(date) 3 October 2012
*We also certify that written approval has been obtained for any proprietary material
contained within.
3
Table of Contents
List of Figures ..................................................................................................................... 5
Chapter 1 – Introduction ................................................................................................... 12
Chapter 2 – Literature Review .......................................................................................... 18
Chapter 2.1 – Insect Models .......................................................................................... 23
Chapter 2.2 – Robot Models ......................................................................................... 25
Chapter 3 – Simulation Environments and Models .......................................................... 29
Chapter 3.1 – Animatlab and Supplementary Environments ........................................ 29
Chapter 3.1.1 – Mechanical Simulations and Models ............................................... 29
Chapter 3.1.2 – Muscle Model .................................................................................. 33
Chapter 3.1.3 – Neuron and Synapse Models ........................................................... 34
Chapter 4– Robust Robotic Stepping ................................................................................ 43
Chapter 4.1 – Stepping Rules ........................................................................................ 43
Chapter 4.2 – Implementation of Stepping Rules ......................................................... 46
Chapter 4.2.1 – Sensory Information ......................................................................... 46
Chapter 4.2.2 – Sensory Interneurons and Reflex Reversal ...................................... 50
Chapter 4.2.3 – Central Pattern Generators ............................................................... 51
Chapter 4.2.4 – Muscle Control Units ....................................................................... 55
Chapter 4.3 – Networks and Their Function ................................................................. 61
Chapter 4.3.1 – Middle Leg Network ........................................................................ 61
Chapter 4.3.2 – Front Leg Network ........................................................................... 63
Chapter 4.3.3 – Hind Leg Network ........................................................................... 64
Chapter 4.4 – Stepping Results ..................................................................................... 66
Chapter 4.4.1 – Stepping Robustness ........................................................................ 66
Chapter 4.4.2 – Comparison to Blaberus .................................................................. 70
Chapter 4.5 – Robotic Implementation ......................................................................... 74
Chapter 5 – Smooth Low Level Transitions ..................................................................... 78
Chapter 5.1 – Implementing Behavior Changes via Reflex Reversals ......................... 79
Chapter 5.2 – Flexible Networks Capable of Changing Gait ........................................ 80
Chapter 5.2.1 – Gait Changes in the Middle Leg ...................................................... 81
4
Chapter 5.2.2 – Gait Changes in the Front Leg ......................................................... 84
Chapter 5.3 – Effect of CPGs on Gait Transitions ........................................................ 84
Chapter 6 – Smooth Behavioral Changes ......................................................................... 88
Chapter 6.1 – Intermediate Level Coordination ............................................................ 88
Chapter 6.2 – Intermediate and Low-Level Gait Changes ............................................ 90
Chapter 6.2.1 – Changing Intermediate Gait ............................................................. 92
Chapter 6.2.2 – Changing Low Level Gait ................................................................ 93
Chapter 7 – Conclusions and Future Work ....................................................................... 97
Chapter 7.1 – Conclusions ............................................................................................ 97
Chapter 7.2 – Future Work ............................................................................................ 98
Chapter 7.2.1 – Sensitivity Analysis and Parameter Tuning ..................................... 98
Chapter 7.2.2 – Actuator Types ............................................................................... 100
Chapter 7.2.3 – Intermediate Circuit ....................................................................... 100
Chapter 7.2.4 – Robotic Leg .................................................................................... 101
Appendix A – Network Topologies ................................................................................ 103
Front Leg ..................................................................................................................... 104
Middle Leg .................................................................................................................. 106
Hind Leg ...................................................................................................................... 108
Intermediate Level Circuit .......................................................................................... 110
Bibliography ................................................................................................................... 113
5
List of Figures
Table 1 - Table of stepping rules discovered in insects, how they are modified in
SimROACH, and the original source................................................................................ 22
Figure 2 – Microscopic photographs of severed cockroach legs (left) for comparison to
triangulated meshes (right) used in simulation. ................................................................ 31
Figure 3 – Mechanical equivalent model to the linear Hill muscle model (top). Length-
tension relationship that limits tension output of the muscle (bottom left). Stimulus-
tension relationship that determines the activation of the muscle (bottom right). Taken
from (Shadmehr and Arbib 1992), accessed on Animatlab.com. ..................................... 33
Figure 4 – Plots showing the step response of neurons with different spiking threshold
accommodation values. An accommodation of 0 makes the spiking frequency a function
of stimulus current (top). An accommodation of 1 makes the spiking frequency a function
of the derivative of the stimulus current (middle). A value between those will produce a
response that is a combination of the two (bottom). The stimulus current (green) is 10 nA
in every picture. ................................................................................................................ 37
Figure 5 – Postsynaptic response of a neuron coupled to a tonically firing neuron via a
synapse with a facilitation of 1 (top) and 0.5 (bottom). Facilitation values less than one
cause the synapse to decay at a rate that is a function of presynaptic spiking frequency. 38
Figure 6 – Plots that show the nonspiking neuron’s step response without calcium
currents (top), and with calcium currents during activation (middle) and deactivation
(bottom). All stimuli (green) have a magnitude of 10 nA. ............................................... 41
Figure 7 – Table of sensory triggers used to generate forward walking in the middle leg.
........................................................................................................................................... 44
Figure 8 – Table of sensory triggers used to generate forward walking in the front leg. . 45
Figure 9 – Plots showing the measured angle (top), transduced current (middle), and
resulting voltage of a neuron coding for the joint’s rotation (bottom). The gray lines show
that the integration lag between the current and the neuron’s voltage is virtually
nonexistent. ....................................................................................................................... 47
Figure 10 – Plots showing the measured angle (top) and the current injected into the rest
of the system to signal that an extreme position has been reached (bottom). There is no
output until the joint reaches a certain limit, at which point it rapidly increases. The gray
lines show what FTi angles signal to the rest of the network. .......................................... 48
Figure 11 – (Top) Unpublished results from the Zill lab showing how some populations
in the cockroach respond to increasing load while others respond to decreasing load.
(Bottom) Picture of network that processing loading information. Neuron A turns the
signal D, the magnitude of the load on the foot, into a firing frequency (E.). Neuron B is
inhibited by neuron A, and will fire when the load goes away. Neuron C is stimulated by
6
Neuron B with a decaying synapse, so its activity decays rapidly when the load ends (F.).
........................................................................................................................................... 49
Figure 12 – Schematic of how a reflex reversal can be executed in this model. The Gait
neuron can affect interneurons that relay sensory information, changing which neurons
are affected by which sensors. .......................................................................................... 50
Figure 13 – Voltage of one half-center of a CPG during oscillation. The oscillation
reaches steady state after about 1 second. ......................................................................... 52
Figure 14 – Schematic of a CPG used in this model. Neurons 1 and 3 are the half-centers,
communicating through interneurons 2 and 4. ................................................................. 52
Figure 15 – Voltage of one half-center of a CPG during normal activity (top) and when
the interneurons are inhibited by a current of -1.25 nA. ................................................... 53
Figure 16 – Voltage of one half-center of a CPG when the presynaptic neuron is strongly
hyperpolarized at different points in the phase. A strong enough stimulus will reset the
phase of the CPG at any point of the phase. The bars along the bottom show the bursting
period before the stimulus was applied. A red circle is drawn around the perturbation,
after which a normal looking period of activity is observed. ........................................... 54
Figure 17 – Behavior of the abstracted muscle pair system. If both stiffnesses (1,1) and
set points (1,-1) are identical and the sin terms are removed, any initial condition will
drive the system toward an angle between the two set points (A). Changing the stiffness
of one set point (k1=5*k2), the system will move toward that set point (B). If the sin
terms are included, the system can be tuned to oscillate with the desired frequency and
amplitude (C). If the frequency is changed, the amplitude is decreased, since this system
is a filter (D). The desired amplitude can be regained by increasing the stiffness of both
set points (E). This is not something the current model is capable of. Keeping the original
stiffnesses and instead increasing the set points will produce qualitatively similar
behavior (F). This is the approach that the current model uses to increase the stiffness of a
joint. .................................................................................................................................. 58
Figure 18 – Plots of joint angles (blue) and extreme position neuron voltages (green).
Note that flexion can be changed independently of extension (top) and vice versa
(middle). They can also be changed together to change the mean angle (bottom). .......... 59
Figure 19 – Schematic of the CPG and muscle control unit for the CTr joint. The CPG
(red) only inhibits the Inter Pos neurons, which are interneurons between the error
feedback control for each muscle and its motor neuron. .................................................. 60
Figure 20 – Control network for the middle leg of the cockroach with no particular gait
active (top) and with the forward walking gait active (bottom). The inactive pathways
have reduced fill. ............................................................................................................... 62
7
Figure 21 – Control network for the front leg of the cockroach with no particular gait
active (top) and with the forward walking gait active (bottom). The inactive pathways
have reduced fill. ............................................................................................................... 64
Figure 22 – Control network for the hind leg of the cockroach. This network is much
smaller than the others because no reflex reversals take place. ........................................ 65
Figure 23 – Three plots showing kinematics during walking in a middle leg without
CPGs and under normal load (top), without CPGs during weighted walking (middle), and
with CPGs during weighted walking (bottom). The leg is able to walk under normal
conditions, but adding extra weight stops the reflex cascade. Adding CPGs to the model
restores rhythmic behavior. The extra inertia causes high frequency noise in the
kinematics that would otherwise be absent. ...................................................................... 69
Figure 24 – Plots showing kinematics during walking for a middle leg without feedback
from one joint in a model without CPGs (top) and a model with CPGs (bottom). A CPG
at every joint reduces the robot’s reliance on sensory information in case of a
malfunction. ...................................................................................................................... 70
Figure 25 – Joint angles of the front leg during tripod walking. The kinematics of the
animal (left) were recorded by Brown on an oiled plate. SimROACH’s kinematics (right)
are provided for comparison. The vertical axes are scaled to match biological data, and
are the same in both graphs. Stance phase is indicated by gray shading. ......................... 71
Figure 26 – Joint angles of the middle leg during tripod walking. The kinematics of the
animal (left) were recorded by Brown on an oiled plate. SimROACH’s kinematics (right)
are provided for comparison. The vertical axes are scaled to match biological data, and
are the same in both graphs. Stance phase is indicated by gray shading. ......................... 72
Figure 27 – Plots comparing muscle activations with the onset of stance in Blaberus
discoidalis (top) and SimROACH (bottom). In both systems the CTr joint is depressed to
cause stance, which causes the extension of the FTi joint. The biological data was
produced by the Zill lab. Stance is indicated in the bottom plot by gray shading. ........... 72
Figure 28 – Joint angles of the hind leg during walking. The kinematics of the animal
(left) were recorded by Brown on an oiled plate. SimROACH’s kinematics (right) are
provided for comparison. The vertical axes are scaled to match biological data, and are
the same in both graphs. Stance phase is indicated by gray shading. ............................... 73
Figure 29 – Picture of the robotic leg used for hardware testing (A). It manages input and
output through a NI CompactRIO (B) and outputs data to LabView (C). ........................ 75
Figure 30 – Joint Angles (top) and CPG activity (bottom) from a walking trial performed
with the robotic leg. Stance is indicated by gray shading. ................................................ 76
Figure 31 – Diagrams that explain LegConNet when producing forward (left) and inside
turning forward (right) behavior. Gait changes are generated by changing the connections
8
and thresholds between sensory influences and bistable “CPGs”. Taken with permission
from (B L Rutter et al. 2011) ............................................................................................ 78
Figure 32 – Tables that show stepping rules for inside turning (top) and outside turning
(bottom) implemented in the middle leg of this model. There is no one authoritative
source for these turning rules, but they are based on literature and hypothesized
transitions. ......................................................................................................................... 81
Figure 33 – Control networks for inside turning (top) and outside turning (bottom) in the
middle leg model. The sensory pathways are highlighted to match the rules listed in
Figure 32. The behavior changes are the result of rerouting sensory information and
turning CPGs off where necessary. ................................................................................... 82
Figure 34 – CPG output from the middle leg during the transition to inside turning (top)
and outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by
pink shading. ..................................................................................................................... 83
Figure 35 – Tables that show stepping rules for inside turning (top) and outside turning
(bottom) implemented in the front leg of this model. There is no one authoritative source
for these turning rules, but they are based on literature and hypothesized transitions. .... 84
Figure 36 – Control network for the front leg configured to generate inside turning (top)
and outside turning (bottom). The inactive pathways have been only partially filled. The
rules for these networks are listed in Figure 35. ............................................................... 85
Figure 37 – CPG output from the front leg during the transition to inside turning (top) and
outside turning (bottom). Stance is indicated by gray shading. Turning is indicated by
pink shading. ..................................................................................................................... 86
Figure 38 – Plots showing how the command to flex the FTi joint (green) is only caused
by loading (blue) in the model without CPGs (top), but can precede loading in the model
with CPGs (bottom). Loading then reinforces this transition, making stepping even more
robust................................................................................................................................. 87
Figure 39 – Intermediate level circuit configured to produce a wave gait (A) and a tripod
gait (B). Inactive pathways are shown with less fill. Synapses are color coded according
to the key at the bottom. .................................................................................................... 89
Figure 40 – Plots showing CPG activity in the three legs on one side while walking with
a wave gait (top) and a tripod gait (bottom). The demonstrated patterns are consistent
with gaits seen in insects. .................................................................................................. 91
Figure 41 – CPG activity during the transition from a wave gait to tripod gait in
ipsilateral (top) and contralateral (bottom) legs. The first trace is the same in each plot.
Tripod walking and the transition are highlighted in pink. ............................................... 92
9
Figure 42 – Picture of a segment of the intermediate circuit configured to turn right by
stimulating the Turn Right neuron, which in turn stimulates the proper low level turning
neurons. ............................................................................................................................. 93
Figure 43 – Plots of CPG activity during the transition from forward walking to turning
while using the wave gait (top) and the tripod gait (bottom). Turning is highlighted in
pink. Dotted lines show that coordination is maintained during the transition. ............... 94
Figure 44 – Robot heading (top) during two typical turning trials. The robot is
commanded to walk straight for 5 s (blue) and then turn (green). The paths were
smoothed with a Gaussian kernel, and the curvature (bottom) for each trial was calculated
as a function of path length. In the left turn trial, the RMS curvature was 5.484 during
forward walking and 28.83 during turning. In the right turn trial, the RMS curvature was
6.317 during walking and 25.01 during turning. .............................................................. 95
Figure 45 – Intermediate level circuit modified to require loading information to tell the
ipsilateral leg to unload. This sensory information is only utilized during the metachronal
wave gait. .......................................................................................................................... 96
10
Acknowledgements
This work, although it has my name on it, was a group effort. Two years ago I
knew nothing about biology and was completely unfamiliar with the processes of
research. Those around me have helped me learn and grow to make this project a reality.
I must thank Dr. Quinn and Dr. Ritzmann for being talented and patient
instructors and mentors. They have given me opportunities, criticisms, and
encouragement that have improved my work and work ethic. I would also like to thank
Dr. Branicky for his insight on this thesis and the future work on this project.
I must thank everyone in the Biologically Inspired Robotics Lab, especially the
Borg Cluster: Hunt for his methodical and thorough thought process, Lonsberry for his
inspiring sense of creativity, BRTietz for his endless knowledge of everything, and
Vickie for her fierce but friendly competition and motivation. Entering the lab with such
a great support group has made this entire experience positive.
I must thank my friends and family for being supportive during this particularly
busy time of my life.
11
Abstract
Massively Distributed Neuromorphic Control for Legged Robots Modeled After Insect
Stepping
By
NICHOLAS STEPHEN SZCZECINSKI
Simulated RObot exhibiting behAvior CHanges (SimROACH) is a massively
distributed control architecture for legged robots composed of simulated physiological
neuron and synapse models. Its structure is based on insect neurobiology. Each joint uses
a unique central pattern generator (CPG) to produce oscillation. The CPGs in each leg
cannot directly communicate, but are coordinated by sensory influences, producing
stepping motion. One CPG from each leg receives input from the same CPG in other
legs, coordinating walking motion. The pathways that coordinate CPGs or legs can be
modified by descending commands to change the way the joints flex or legs step with
respect to one another, smoothly changing gait while in motion. SimROACH walks and
changes gait in a simulated physics environment. SimROACH’s middle leg network was
further verified by successfully controlling a single robotic leg attached to a test stand.
12
Chapter 1 – Introduction
Insects are capable of producing flexible and robust walking motions. They
negotiate different terrains by autonomously adapting their movements to suit their
environment on the fly, something that robots often have difficulty doing correctly. This
flexibility comes from the components and organization of their nervous systems, which
are highly distributed and plastic. By stimulating or inhibiting part of the nervous system,
the qualitative behavior of the entire insect changes, an approach that is rarely taken in
robotics. This thesis presents Simulated RObot exhibiting behAvior CHanges
(SimROACH), a control architecture for legged robots composed of simulated
physiological neuron and synapse models. As is hypothesized in insects, a few
descending influences can reverse reflexes and modify interleg coordination pathways in
SimROACH to change gait and other behaviors in mid-motion. To the author’s
knowledge, this work is unique in the fields of both biology and robotics. While
impressive computational neuroscience models of legged locomotion control exist, no
known computational neuroscience model incorporates the level of mechanical dynamics
as in this simulation, Furthermore, there are no known robots or simulations of robots
controlled with a computational neuroscience with this level of fidelity.
This research has two primary goals and two secondary goals. The primary goals
are to achieve robust robot walking through a control system based in insect
neurobiology, as well as smooth behavior transitions with the same system. Cockroaches
are some of the most agile hexapods, and a robot that could move like a cockroach would
be extremely effective. SimROACH achieves robust walking by mimicking both the
connections of the nervous system and mathematics of individual neurons. The entire
system is controlled by computational neural models connected in ways suggested by
13
insect neurobiology literature. Each leg can walk while the body’s height is changed,
while stepping through holes, when weighted, and when some sensory information is
eliminated. In addition, the front and middle legs can smoothly change between forward
walking, inside turning, and outside turning motions, allowing SimROACH to change its
heading while walking.
SimROACH’s behaviors are based on those observed in cockroaches. Direct
comparison with cockroach movements is instructive because data for that model
organism is available. Therefore SimROACH’s secondary goal is to be a useful model of
insect locomotion. Every piece of SimROACH’s controls is taken directly from the
literature or an educated guess based on biological hypotheses. SimROACH is able to
produce some interesting behaviors despite being an incomplete model. Future work
includes incorporating more control structures from neurobiology.
Much research has been done to determine what causes coordinated stepping in
insects (Akay et al. 2001; Bucher et al. 2003; Akay et al. 2004; Ridgel et al. 1999; Zill,
Keller, and Duke 2009; Zill et al. 2011; Zill, Schmitz, and Büschges 2004). Chapter 2
reviews the literature on the subject and the following in this chapter is an introduction to
that topic. Many ingenious experiments have revealed what sensory cues coordinate the
motion of multiple joints (Akay et al. 2004). The middle leg is often the focus of these
studies because its function is the most general of insect legs.
The result of these experiments has been a more accurate understanding of how
insect motor nervous systems function. One of the key findings is that each joint appears
to be driven by its own central pattern generator (CPG) (Ryckebusch and Laurent 1993;
Büschges, Schmitz, and Bässler 1995). Rather than any one central structure coordinating
14
in what fashion the different segments of the leg move, each can produce its own
rhythmicity. In addition, many nonspiking neurons surrounding the rhythm generators
help convert a rhythmic pattern into coordinated muscle contraction (Büschges 1995).
Some interneurons receive drive from the CPG, while others show activity that is not
affected or in anti-phase. Stimulating some of these neurons will cause the cycle phase to
reset. The rate of oscillation can also be modulated, changing the overall frequency,
changing the activity symmetry, or ceasing oscillation completely (Daun-Gruhn 2010).
This massively distributed structure in insects is crucial to how SimROACH works.
These CPGs are not coupled by direct connections (Büschges, Schmitz, and
Bässler 1995), but rather by sensory information (Büschges et al. 2008; Akay et al. 2001).
These influences are often described as reflexes that cause a change in a joint’s timing
(for example, from flexion to extension), but these influences likely affect CPG timing
(Akay et al. 2004; Büschges et al. 2008). Other simulations have shown that this sensory
coupling is an effective means by which to coordinate CPGs (Daun-Gruhn and Tóth
2010; Spardy et al. 2011; Daun-Gruhn 2010; Ekeberg, Blümel, and Büschges 2004).
Chapter 3 – Simulation Environments and Models describes the computational models
used to simulate these neural populations, and Chapter 4– Robust Robotic Stepping
explains how SimROACH uses these rules and structure to coordinate stepping.
Since the joints in one leg are coordinated into stepping motion by sensory
influences alone, changing where the sensory information goes can change the behavior
of the leg. For instance, switching from forward to backward walking in the stick insect is
the result of only changing what sensory cue causes raising and lowering of the leg (Akay
et al. 2007). Other reflex reversals related to standing still (Akay and Büschges 2006) and
15
turning (Ekeberg, Blümel, and Büschges 2004) have been identified. In addition to neural
connections, muscle activity (Mu and Ritzmann 2005) and joint kinematics (Brown 2011)
are known to change when the cockroach Blaberus discoidalis produces sideways
stepping motions. These rules have been successfully implemented in robotic models of
insect legs capable of stepping according to walking rules (Lewinger and Rutter 2006)
and changing these rules to produce turning behavior (Rutter et al. 2011). Chapter 5 –
Smooth Low Level Transitions explains how SimROACH makes low level network
changes to change gait.
In addition to low level activity and changes, intermediate level coordination has
also been the focus of research. The Cruse rules provide observed rules for coordinating
legs in various arthropods (Cruse 1990). Oil plate experiments with cockroaches and
stick insects suggest that these coordination rules are enforced by neural connections
between the legs rather than purely sensory or mechanical influences (Brown 2011;
Gruhn, Zehl, and Büschges 2009). However, other work has shown that mechanical
linking between legs also has an effect, perhaps reinforcing the neural pathways (Zill,
Keller, and Duke 2009; Ridgel et al. 1999). Models that incorporate both means of
coordination have been shown to successfully coordinate stepping in multi-leg models
(Daun-Gruhn and Tóth 2010; Cruse et al. 1998), suggesting that intermediate
coordination is indeed due to a combination of neural and sensory signals.
Some robots have made use of more abstracted biological principles to produce
effective interleg stepping. Robot II used a finite state machine implementation of
generalized Cruse rules with additional sensory driven leg reflexes to produce robust
stepping on irregular terrain (Espenschied et al. 1995). Other robots in the Biologically
16
Inspired Robotics Laboratory have sought to produce improved results with more
accurate biological structures (Lewinger and Rutter 2006; Lewinger and Quinn 2010;
Rutter, Taylor, and Bender 2011). More recently, the Buschges group has produced a
robot Octavio controlled by artificial neural networks based on stick insect neurobiology.
The neuron models used are abstracted, and the joints do no possess oscillating CPGs, but
the network topologies are based on stick insect pathways (Von Twickel, Büschges, and
Pasemann 2011; von Twickel et al. 2011). Not only are these robots excellent walkers,
but they also replicate the results of stick insect walking experiments.
None of these other systems, however, possess the behavioral flexibility of
SimROACH, which is examined in Chapter 6 – Smooth Behavioral Changes. Its motion
is not identical to that of an insect, but it generates stepping and changes gaits in a similar
manner. The massively distributed control architecture changes behavior through
descending commands that change how sensory information affects each joint. While this
system is unorthodox in the robotics community, initial tests with a robotic platform have
shown that this system is indeed effective at generating stepping. The implementation in
the robot model of a cockroach leg does not currently have the capability to change gait,
this will be added in the near future.
SimROACH is also novel in computational biology in that there is no simulation
that is as complete. Not only does SimROACH simulate interjoint and interleg neural
connections, it also simulates body dynamics and interaction with the environment. Even
though some models use more sophisticated neuron models (Daun-Gruhn 2010; Daun-
Gruhn and Tóth 2010), they and others do not simulate the kinetics of the animal (Cruse
et al. 1998; Ekeberg, Blümel, and Büschges 2004). This trend is only now changing in
17
the field of computational neuroscience (Tóth, Knops, and Daun-Gruhn 2012).
SimROACH is not a complete simulation of a cockroach, but the simulation is the most
holistic to the author’s knowledge.
18
Chapter 2 – Literature Review
Insect locomotion has been an active area of research for many decades. Much
work has been done to understand how the multiple joints of each leg are coordinated
into walking motion, how these joints change their motion to generate different gaits, and
how legs communicate with one another. This research may be classified into behavioral,
neural systems, or a combination of the two. The work has led to finite state machine and
neural models of control systems that have sometimes been implemented in legged
robots. Much of it has been conducted on stick insects in Europe and cockroaches in the
United States. SimROACH’s structure draws elements from both organisms since it is
generally accepted that rules from one of these animals can be adapted to the other.
SimROACH accomplishes its primary goals of robust walking and smooth
behavior changes by mimicking what is known about insect locomotion. In addition,
SimROACH’s secondary goal is to be a useful model of insect locomotion. Therefore
knowledge of insect neurobiology was crucial to its development. Most of the specific
sensory-motor interactions incorporated into SimROACH come from results published
after 1995. Many studies before this point revealed behavioral patterns, but the research
described below identified specific neural systems and cause and effect relationships
between sensors and muscles. Büschges, Schmitz, and Bässler (1995) found evidence that
each joint in the stick insect leg is controlled by its own CPG. When deaffarented (i.e. all
incoming connections were removed) and subjected to pilocarpine (i.e. an M-receptor
agonist, mimicking acetylcholine), each joint’s motor neurons fired rhythmically, but
uncoordinated with other joints. This suggested that walking is due to the assembly of
modular units, that is, independent joint controllers. These rhythms are coordinated in the
intact animal by nonspiking sensory interneurons.
19
Hess and Büschges (1999) found that in the stick insect, the angle of the femur-
tibia (FTi) joint affected when the coxa-trochanter (CTr) joint extended or flexed in the
stepping cycle. Bucher et al. (2003) found that these signals were more influential at
extreme angles, and could cease CPG oscillation.
Load sensors also contribute to coordinated stepping (Akay et al. 2001). Signals
from the femoral campaniform sensilla (fCS) are important in generating motion in the
FTi joint. Unloading the leg by flexing the CTr joint causes the FTi joint to extend, but
loading the leg by extending the CTr joint does not cause the FTi joint to flex. Later work
showed that the trochanteral campaniform sensilla (trCS) are more important for
maintaining stepping coordination, while the fCS is important to controlling the FTi joint
itself (Akay et al. 2004).
“Peg leg” experiments in stick insects reinforced the observation that the trCS are
most important for signaling leg loading. When the middle leg is deaffarented and
deefferented distal from the middle of the femur, stick insects can walk in a normal
fashion (Noah et al. 2004). This is consistent with the observations of Akay et al. (2001),
which suggested that the other campaniform sensilla modulate FTi muscle strength, and
thus were not important to peg leg walking.
Further research on load sensors reaffirmed that loading modulates both muscle
activity timing and strength throughout stepping (Zill, Schmitz, and Büschges 2004). CS
are able to provide the rest of the nervous system with quite sophisticated input, including
signals that respond to load magnitude, load direction, and the rates of increase and
decrease of the load (Zill, Büschges, and Schmitz 2011). Also, some populations were
active when the CS were unloaded (Ridgel et al. 1999). This means the lack of load
20
actively generates a sensory signal, which is a stronger influence on the network than the
lack of a loading signal.
These sensory influences were known to alter CPG timing among different joints,
but how does CPG activity affect muscle contraction? Büschges et al. (2004) showed that
when CPG activity was suppressed by hyperpolarizing current, motor neurons were more
excited. When enough current to halt oscillation was applied to a CPG, the associated
motor neurons depolarized to a constant value. This suggests that motor neurons are
under a constant tonic drive, and CPGs suppress them rhythmically, rather than exciting
them.
Other work has sought to explain other types of leg behavior, such as standing
still, walking backward, or turning. This has led to data on behavioral changes, as well as
hypotheses as to how this occurs. (Mu and Ritzmann 2005) used kinematics and muscle
activity to show that cockroaches change how they use their front four legs during
turning, appearing to reverse key reflexes. Research in stick insects has produced similar
results, showing that turning behavior is a low-level change (Hellekes et al. 2012).
Further work with cockroaches has detailed the kinematic changes that occur in each
joint of the organism while walking forward and turning (Brown 2011).
Work in forward and backward walking stick insects revealed other reflex
reversals and provided hypotheses about how these changes might occur in the nervous
system (Akay et al. 2007). This work showed that altering the stepping motion of a single
leg is a low level change, and can be produced by changing the order or direction of each
joint’s motion with respect to the others. The authors hypothesize this can be
21
accomplished by altering synaptic weights among parallel sensory influences, changing
which sensory influences cause which joint to move.
Interleg interactions have also been a topic of research. The Cruse rules are
sufficient to coordinate multiple legs (Cruse 1990). These are based on behavioral
experiments performed on various arthropods and have served as the basis of many
animal models and robots (Lewinger and Quinn 2010; Cruse et al. 1998; Daun-Gruhn
2010; Beer et al. 1992; Espenschied et al. 1995; Nelson et al. 1997). These rules specify
that loading a leg will promote unloading of the anterior leg, unloading a leg will prevent
unloading of the anterior leg, leg stepping is targeted to track the successful placement of
the anterior leg, legs will restep if they collide with the anterior leg, and loading the
organism will increase the time spent in stance.
The Cruse rules do not provide a mechanism, either neural or mechanical, for such
coupling. Hypotheses and models based on proprioception and direct neural connections
have been proposed. Experiments in stick insects suggest that the angle of FTi flexion
caused transitions between stance and swing in adjacent legs (Bucher et al. 2003). Other
experiments have shown that the onset of stance in one leg reduces the load in the
anterior leg, promoting the transition to swing (Zill, Keller, and Duke 2009). Models of
such coordination, however, typically use sensory information to modulate direct
connections between CPGs to maintain coordination (Cruse et al. 1998; Daun-Gruhn
2010).
22
Nearly all of these rules related to stepping have been implemented in
SimROACH. Table 1 lists specific features, how they may be adapted to the cockroach,
and the source from which each came. These features can be compared to prominent
Table of Insect Stepping Rules and Their Adaptation to SimROACH
Biological Observation Modifications in SimROACH Original Publication
Each joint in the leg has its own CPG
with no direct connections to the others
None (Büschges, Schmitz, and
Bässler 1995)
FTi joint angle affects CTr motion Direction of FTi movement
reversed with respect to CTr in
cockroach
(Hess and Büschges 1999)
FTi joint angle more strongly affects CTr
motion at extreme angles
None (Bucher et al. 2003)
fCS affects motion of the FTi joint All loading detected by tarsus (Akay et al. 2001)
trCS affects timing of the FTi joint All loading detected by tarsus (Akay et al. 2004)
trCS is the main input for determining
stance
All loading detected by tarsus (Noah et al. 2004)
Loading modulates muscle activity
throughout the leg during stance
All loading detected by tarsus (Zill, Schmitz, and
Büschges 2004)
CS code for load, rate of increase of load,
rate of decrease of load
None (Zill, Büschges, and
Schmitz 2011)
CS code for unloaded None (Ridgel et al. 1999)
Rhythmic muscle activity is due to CPG
suppression of otherwise excited motor
neurons
None (Büschges et al. 2004)
Strong hyperpolarizing stimulus to a CPG
will halt oscillation
None (Büschges et al. 2004)
Muscle activity, joint angle ranges, and
joint angle phases change in the CTr and
FTi joints of cockroaches while turning
Amplitude of changes do not
precisely match observations in
cockroach
(Mu and Ritzmann 2005)
Turning is the result of low level reflex
changes only
None (Hellekes et al. 2011)
Turning behavior can be classified by
different joint kinematics than walking
None (Brown 2011)
Gait changes (forward and backward
walking) are due to reversing low level
reflexes
None (Akay et al. 2007)
Loading a leg excites unloading the
anterior leg; Unloading a leg inhibits
unloading in the anterior leg; Loading a
leg excites unloading in the contralateral
leg; Unloading a leg inhibits unloading in
the contralateral leg
Direction of ipsilateral coupling
reversed (that is, from front to
back)
(Cruse 1990)
Legs may be coupled by sensory gated
connections between one CPG in each leg
Connections do not include
sensory gating
(Daun-Gruhn 2010)
TrF joint active in middle and hind legs
during cockroach walking
None (Bender, Simpson, and
Ritzmann 2010)
TrF joint only actively extended TrF joint both actively flexed and
extended
(Carbonell 1947)
Table 1 - Table of stepping rules discovered in insects, how they are modified in SimROACH, and the
original source.
23
computational biology projects: Walknet (Cruse et al. 1998), the Ekeberg leg simulation
(Ekeberg, Blümel, and Büschges 2004), and the work of Silvia Daun-Gruhn (Daun-
Gruhn 2010; Daun-Gruhn and Tóth 2010).
Chapter 2.1 – Insect Models
The Walknet simulation is an artificial neural network that coordinates the joints
and legs of a stick insect simulation into walking motions. Each of the legs can be in
either of two states, stance and swing, at a time. These states select a positional controller
that moves each joint toward extreme positions. These are implemented as feedforward
perceptron networks that take in joint angles and output joint velocities. These networks
can be modified to produce turning motions. In addition, the legs can be coordinated into
a tripod or tetrapod gait in a kinematic simulation. These capabilities are impressive, but
SimROACH differs in some key ways. SimROACH can produce the same types of
behavior as Walknet (walking in tripod or tetrapod gaits and turning), but all mechanical
dynamics are simulated. In addition, SimROACH uses dynamical neurons, not static
“neuroids.” This allows SimROACH to use dynamical, naturally oscillating CPGs at each
joint. Such a feature helps improve the stability of walking motion, particularly while
changing gaits as discussed in Chapter 5.3 – Effect of CPGs on Gait Transitions.
The Ekeberg simulation is a finite state machine that uses rules from Akay et al.
(2004), Akay et al. (2001), Bucher et al. (2003), Hess and Büschges (1997), and others to
simulate walking in a dynamical simulation of all legs of the stick insect. Rules that
generate stepping in the stick insect middle leg were adapted to the front and hind legs.
Each joint’s direction of motion was controlled by a bistable element whose state could
be changed by the proper sensory cue. Each leg was simulated in software, but only one
24
was active at a time. The single legs were then used to repeat experiments performed on
actual organisms. Results from both restricted stepping and single leg stepping were
replicated, showing the merit of such modeling work. The model, however, was
admittedly simple, and the authors hoped that muscle models and magnitude control of
the muscles would produce a more realistic model. These are both features that
SimROACH incorporates. In addition, SimROACH uses dynamical CPGs, neurons and
synapses instead of bistable elements in a finite state machine and coordinates multiple
legs stepping at the same time. The Ekeberg et al. simulation, however, was very useful
to the development of SimROACH in that the stepping rules were able to be adapted to
control the cockroach.
A higher fidelity simulation of the stick insect nervous system is that presented by
Silvia Daun-Gruhn in (Daun-Gruhn 2010; Daun-Gruhn and Tóth 2010). The simulation
controls stepping in a full stick insect model with a network of Hodgkin-Huxley type
neuron models configured to follow the rules from (Ekeberg, Blümel, and Büschges
2004). Instead of bistable units, this model is the first to simulate an independent
dynamical CPG controlling every joint of the animal. Not only are these CPGs coupled
within each leg via sensory influences, but they also communicate with those in other
legs through sensory-gated connections between CPGs. All connections but a few are
based on known neural connectivity in the stick insect. SimROACH, like the Daun-
Gruhn model, has a unique CPG controlling the timing of each joint, coupled to the
others only through shared sensory influences. In addition, legs are coordinated with one
another by coupling the one CPG from each joint to the others. The Daun-Gruhn model,
however, uses a more accurate neuron model. SimROACH uses a simpler model for
25
computational efficiency because it is intended to eventually run in real time on board a
robot, whereas Daun-Gruhn’s models are biological tools and do not need to simulate
quickly. SimROACH, however, does have some advantages. For instance, it possesses
both CPG timing control and sensory feedback magnitude control of muscles, rather than
just CPG timing control as in the Daun-Gruhn model. This makes stepping more
adaptable, something that is perhaps more important to a robot. In addition, the Daun-
Gruhn model does not make low-level network changes to produce different motions like
turning or reaching, although the authors of (Daun-Gruhn 2010) suggest that this would
be possible given the network topology (something that SimROACH validates). Finally,
SimROACH exists in a fully simulated dynamical environment, whereas mechanical
dynamics are currently being integrated into the Daun-Gruhn model in pieces (Tóth,
Knops, and Daun-Gruhn 2012).
Chapter 2.2 – Robot Models
Although SimROACH can be classified as a biological model, its primary goal is
to produce robust walking and smooth behavior transitions. Since it was developed for
use in robotics, it should be compared to other legged robot control schemes. There have
been many distributed control networks based in biology, so the goal is for SimROACH
to offer advantages over these alternatives.
Robot I (Beer et al. 1992) used a distributed neural system capable of generating a
variety of gaits and stepping through a variety of environments. Robot I’s control system
was based on literature describing observed insect behavior (Beer et al. 1992). Robot II
had a distributed control system with additional leg reflexes that made stepping extremely
adaptable (Espenschied et al. 1996). These robots are exceptional walkers, and
26
SimROACH does not possess all of their capabilities. SimROACH currently cannot
produce motions that are as adaptable as Robot II’s because it lacks some of Robot’s
reflexes, but it uses actual pathways identified in insects to generate movement.
Therefore, its success is encouraging in that future work implementing more details from
animal nervous systems may lead to more animal like agility and robustness that
surpasses current robot capabilities.
The principles that made Robot I and Robot II successful have been used in other
robots since then. Others have been developed that use either abstracted or more
biological neural systems in addition to reflexes to coordinate movement. Tekken2 uses
an artificial neural network to generate rhythms and reflexes for stable all-terrain
quadruped walking (Kimura, Fukuoka, and Cohen 2007). Similarly, AMOS-WD06 uses
an artificial neural network to coordinate walking, negotiate obstacles, and react to light
stimulus (Manoonpong, Pasemann, and Florentin 2007). These robots show that even
abstracted neural systems with the proper reflexes can successfully traverse difficult
terrain.
Further work in the Biologically Inspired Robotics Lab developed the controller
as presented by Ekeberg et al. into a finite state machine called Sensory Coupled Action
Switching Modules (SCASM) (Lewinger and Rutter 2006). This system was used to
control a single leg on a static test stand and a pair of legs on a wheel set. It generated
stepping by recreating much of the system from (Ekeberg, Blümel, and Büschges 2004).
Bistable CPGs were coordinated by sensory information crossing thresholds, and were
used to stimulate simulated muscles. Both platforms were capable of producing stepping
motion, although improper transitions between FTi flexion and extension were observed.
27
Adding another sensory threshold resolved this issue, but increased the complexity of the
system. Tests with SimROACH and a version of SimROACH built to mimic SCASM
suggest that placing a rhythmic CPG at every joint improves stepping stability (See
Chapter 4.4.1 – Stepping Robustness). In addition, a single leg robotic implementation of
SimROACH did not produce any problems with stepping stability.
SCASM was used to produce Leg Control Network (LegConNet), a system that
could change gaits on the fly by reversing reflexes (Rutter et al. 2011). Higher command
centers were used to modify the weights of various sensory pathways to the bistable CPG
units at each joint. When applied instantaneously, the command to change gait
successfully caused new behavior. If the sensory pathways or stepping rules were
changed in a continuous way, the system was not so successful. The authors of (Rutter et
al. 2011) hypothesized that implementing these rules as a neural simulation would
resolve this problem, something that SimROACH validates.
LegConNet was used to control all six legs in BILL-Ant-a (Biologically Inspired
Legged Locomotion – Ant – autonomous) (Lewinger and Quinn 2010). Six such legs
were coordinated by the most essential Cruse rules to generate coordinated stepping. The
gait used by the robot depended on the initial position of each foot, but could exhibit a
wave gait, tetrapod gait, and tripod gait as the speed was increased. LegConNet was
modified to include reflexes to correct stepping, including an elevator reflex and a
searching reflex. This allowed the robot to navigate obstructions. This system is perhaps
the most similar to SimROACH in its goal to produce robust robotic behavior as strongly
based on biology as possible. BILL-Ant-a used finite state machines, not a neural system,
28
but its ability to change speed and its stepping reflexes give it abilities that SimROACH
does not yet possess.
The stick insect rules from (Ekeberg, Blümel, and Büschges 2004) are being
implemented in Octavio, a robot modeled after stick insect, to be used to study insect
neurobiology (Von Twickel, Büschges, and Pasemann 2011; von Twickel et al. 2011). It
has a distributed control system composed of sigmoidal activation neurons. Topologies
based on experiments produced successful walking behavior, and then additional systems
were developed with the help of genetic algorithms. These do not have connectivity
based in experimental results, but produce behavior that is similar to that seen in the
animal. These evolutionary algorithms included routines that could build structures in
addition to tuning parameters, or focus on building local substructures. Algorithms like
these may benefit SimROACH in the future.
This related work has all influenced SimROACH. The biological data in the
literature provided many rules and hypotheses about how motor systems are controlled.
Computational models showed what kind of assumptions are acceptable and what level of
detail is typically accounted for. Other robots highlight issues that need to be addressed in
robotics and serve as performance benchmarks for this work. SimROACH has benefited
from all of the work of these previous projects, and hopes to show that a more
biomimetic control system can produce robust walking as well as smooth behavioral
transitions. SimROACH can also be considered a biological model, so it is constructed of
models used in computational neuroscience. These are detailed in the next chapter.
29
Chapter 3 – Simulation Environments and Models
Simulation was a crucial part of this work. Most of the actual model development
was conducted in Animatlab, an open source neuromechanical simulator (Cofer et al.
2010). Mechanical models of cockroach appendages were developed in Blender
(Stichting Blender Foundation, Amsterdam, Netherlands), an open source triangulated
mesh editor. Neural models from Animatlab were reproduced in Matlab (Mathworks,
Natick, Massachusetts) and XPP (G. Bard Ermentrout, University of Pittsburgh) when
more rigorous mathematical analysis of particular portions of the system was needed.
Chapter 3.1 – Animatlab and Supplementary Environments
Animatlab is a visual C++ Windows application developed by David Cofer of the
University of Georgia as a part of his Ph.D. dissertation. It is a neuromechanical editor
that simulates neural dynamics, mechanical dynamics, and interactions between them. A
variety of neural models is available for use, and can be dragged and dropped into a
connection network. Physics are simulated by the Vortex physics engine (CM Labs,
Quebec, Canada). The two interact through muscles and sensors. Understanding how
these models work is important for grasping the presented work, its capabilities, and its
limitations.
Chapter 3.1.1 – Mechanical Simulations and Models
Animatlab can use triangulated meshes to simulate body inertia and collisions.
The inertia properties of each body are the result of applying a uniform density to the
mesh. This means the user can produce any shape and simulate its translational and
rotational dynamics based on geometry. The legs of Blaberus discoidalis were dissected
and measured for modeling.
30
Measurements were taken with the help of Al Pollack in the Ritzmann Lab.
Female cockroaches judged to be normal in size and appearance were collected and
sedated with carbon dioxide, decapitated, and frozen for approximately fifteen minutes.
This allowed their internal fluids to congeal without reducing the body’s flexibility. The
bodies were pinned to petri dishes filled with silicone and examined under a microscope.
Digital photography and markup were used to save and annotate images of the
decapitated cockroach and its amputated legs in various configurations. Examples of
these can be seen in Figure 2.
These images were used to produce meshes of each of the legs’ segments. The
distal segments (tarsus, tibia, femur, trochanter) were relatively straightforward to model
since it is usually clear where one ends and the next begins. Modeling the coxa, however,
was quite complicated. The front leg possesses three degrees of freedom between its coxa
and thorax, and the other legs possess two. It is very hard to tell where the axis of each
joint lies, its orientation, and its relative order of proximity to the thorax. Rotation
matrices of bodies are order specific, which means that stacking the joints in the wrong
order produces motions that are not comparable to what might be seen in the organism.
31
Figure 2 – Microscopic photographs of severed cockroach legs (left) for comparison to triangulated meshes
(right) used in simulation.
32
The joints were assigned in the order Thorax-coxa (ThC) 2, ThC1, ThC3, Coxa-
trochanter (CTr), Trochanter-femur (TrF), and Femur-tibia (FTi) distal from the thorax.
The TrF joint is locked in the front leg, and the ThC3 joints are not included in the
middle and hind legs (Bender, Simpson, and Ritzmann 2010).
The Tibia-tarsus joint is locked in SimROACH. The tarsus is a compliant,
actuated member that aids in proper foot placement and holding. This feature was omitted
in favor of modeling simplicity and lower simulation run time (fewer actuators/dynamics
to simulate). Cockroach posture reduces the need for foot pressure control, compared to
more upright quadrupeds and bipeds that use foot pressure to maintain balance (Chou et
al. 2009; Meyer, Oddsson, and De Luca 2004).
Figure 2 shows the actuated joints labeled on each leg. The name and orientation
of each joint are labeled. Joint angle measurements were recorded according to (Bender,
Simpson, and Ritzmann 2010). To summarize, extension of each joint is a positive angle.
The CTr and FTi joints would be 180 degrees if extended into straight lines, and flexing
the joints reduces the angle. The TrF joint measures the angle between the plane defined
by the coxa and the plane defined by the femur and tibia. The ThC1 joint is measured
between the outside edge of the coxa and a vertical line pointing below the thorax. The
ThC2 angle is measured between the axis of the ThC1 joint and a line horizontal to the
thorax. Finally, the ThC3 joint is the rotation of the coxa about its own outside edge.
These conventions will be used throughout the rest of this document.
All joints are modeled as one degree of freedom pin joints. Mechanical stops exist
at the limits as determined by dissections. The limits are modeled as damped spring
buffers that are only active past the specified angle.
33
Chapter 3.1.2 – Muscle Model
All of the muscles in SimROACH are a version of the linear Hill muscle model,
which is built into Animatlab.
The Hill muscle model is the
result of force-displacement
experiments performed on
muscle tissue. The collected
data were used to produce a
mechanical equivalent of
springs and dampers. That
system can be seen in Figure
3. There are three passive
components in the model: a
series spring, a parallel spring,
and a parallel damper. The
series spring’s stiffness controls the strength of the muscle, the parallel spring’s stiffness
determines the amount of output energy that is stored per cycle, and the damper keeps the
muscle from contracting too quickly. The tension in the muscle develops according to the
differential equation:
( ̇ (
) )
Where is the tension the muscle applies, , , and are the series stiffness, parallel
stiffness, and damping, respectively, is the length of the muscle, and is the activation
level of the muscle.
Figure 3 – Mechanical equivalent model to the linear Hill muscle
model (top). Length-tension relationship that limits tension output of
the muscle (bottom left). Stimulus-tension relationship that
determines the activation of the muscle (bottom right). Taken from
(Shadmehr and Arbib 1992), accessed on Animatlab.com.
34
The activation level is determined by the voltage of the motor neuron pools in this
model. Each muscle has a sigmoidal transfer function between the voltage of its motor
neuron and the activation of the muscle. The activation level of the muscle is also a
function of its length. Each muscle has a quadratic relationship relating its ability to apply
tension to its length. It is modeled as parabolic because it is a close fit to data collected in
frogs, cats, and humans. This particular model has also been used to model insect
muscles (Cofer 2009). The heuristic for such tuning is that the muscle can apply full
tension at its resting length and no tension at 133% and 67% of that length (Rassier,
MacIntosh, and Herzog 1999). These three points define a parabola. Figure 3 shows plots
of these two additional relationships.
These relationships require proper tuning to produce useful tension. The length-
tension relationship in particular is difficult to manage. If one muscle of an antagonistic
pair applies too much tension, it may pull the other muscle to a length at which it cannot
apply tension and pull the limb back. The time dependent generation of muscle tension
also makes the system more difficult to understand. All of the values in SimROACH
have been set by examining the steady state tension of the muscles when each joint is at
its extreme positions. This was then modified based on actual performance.
Chapter 3.1.3 – Neuron and Synapse Models
Many neuron models exist, and each highlights certain behaviors of neurons. The
controls of SimROACH were constructed from neurons to more directly mimic findings
in insects and hopefully produce more robust walking. However, if this system is to
control a robot in real time, the neuron models must be simple and relatively easy to
simulate. Therefore the integrate and fire (IF) model was selected for this work. It is
perhaps the simplest model that simulates membrane dynamics rather than more
35
abstracted quantities such as firing frequency. This means neural data from SimROACH
can potentially be directly compared to recordings in insects. Both spiking and
nonspiking versions of this model exist, despite the contradictory “nonspiking integrate
and fire” name. A quantitative analysis of the behavior and simulation speeds of different
spiking neuron models is available in (Izhikevich 2004).
Chapter 3.1.3.1 – Spiking Integrate and Fire Model
Only the neurons that code for loading, unloading, and decreasing load are
spiking neurons in SimROACH. This choice was made because it made for more direct
comparisons between recordings from campaniform sensilla and SimROACH. This is
also desirable because the spiking model, unlike the nonspiking model, can accommodate
its spiking threshold and facilitate its synapses. This means properties change over time,
encoding for derivatives or integrals of stimuli. These features were used to more closely
mimic recordings from animals that reveal detection of load and the time derivative of
load.
Despite being simple, the IF model captures the basic characteristics of some
neurons in a mathematically efficient way. The spiking IF model is a leaky integrator that
will depolarize above a certain voltage threshold, then hyperpolarize below the resting
voltage. The rate at which it spikes depends on the current stimulus applied or the rate at
which it changes. These qualitative descriptions are consistent with the observations of
Hodgkin and Huxley, who produced the first detailed description of neural behavior
(Hodgkin, Huxley, and Katz 1952). The manner in which the IF model achieves these
features is much simpler than the Hodgkin Huxley (HH) equations.
The spiking neuron can receive current input from artificial stimuli ( ,
synapses ( , and what is called the after-hyperpolarization (AHP) current. The IF
36
only keeps track of the charge of the current, not particular ions like more sophisticated
models. Therefore a spike is recorded when the voltage crosses the spiking threshold.
Instead of the voltage increasing quickly due to ion channel dynamics, a cosmetic spike is
applied and the AHP current is activated. Because the spike is cosmetic, the calculated
membrane voltage does not change rapidly. This is advantageous because a simple
integration scheme like forward Euler can be used to simulate membrane dynamics.
The AHP current is applied by establishing an exponentially decaying
conductance between the membrane voltage and the AHP voltage ( ), which
hyperpolarizes the neuron and ensures there is time between spikes. This is different from
the canonical integrate and fire neuron model, which sets the membrane voltage to a
specified value in the time step after a spike. Applying the AHP current instead more
closely mimics the shape of hyperpolarization seen after a neuron spikes and produces
more flexible neural behavior.
The IF membrane potential changes according to:
[( ( ) ]
where
{
The neuron leaks according to the membrane conductance ( between the
membrane voltage ( and the resting voltage ( . The AHP current is applied after ,
which is the time of the last spike, and decays with rate . Spikes are recorded when
the voltage crosses the spiking threshold , which accommodates according to:
( ( (
⁄
37
The spiking threshold accommodates from
its initial value an amount proportional
to the amount that the membrane voltage
changes from its resting value. This
proportionality is set by , and the
threshold changes with time constant .
Manipulating and can make the
neuron emulate class III excitability, that is,
activity that corresponds to the rate of
change of the stimulus. Figure 4 shows the
step response of the spiking neuron model
with different values for threshold
accommodation. Note how choosing an
accommodation value between zero and one
produces spiking frequency that is sensitive
to both the stimulation level and the rate of
stimulus.
The synaptic current, , is defined
by:
(
{
can facilitate by:
Figure 4 – Plots showing the step response of neurons
with different spiking threshold accommodation
values. An accommodation of 0 makes the spiking
frequency a function of stimulus current (top). An
accommodation of 1 makes the spiking frequency a
function of the derivative of the stimulus current
(middle). A value between those will produce a
response that is a combination of the two (bottom).
The stimulus current (green) is 10 nA in every
picture.
38
( ∑ (
)
is the static voltage of the synapse, is the conductance of the synapse at a
given instant, is the user-specified synaptic strength, which decays at rate .
This means each spike injects a decaying (roughly) exponential current into the
postsynaptic neuron. The strength of the synapse can change from spike to spike
according to , the synaptic facilitation. If , then each synapse’s conductance is
Figure 5 – Postsynaptic response of a neuron coupled to a tonically firing neuron via a synapse with a
facilitation of 1 (top) and 0.5 (bottom). Facilitation values less than one cause the synapse to decay at a rate
that is a function of presynaptic spiking frequency.
39
lower than the maximum conductance by a factor of (
. As the spikes
continue, the cumulative effect of each is maintained, requiring that the synapse keep
track of when the presynaptic neuron spikes for the duration of the simulation. This is
used to make connections between neurons decay or strengthen. Figure 5 shows the
postsynaptic response of a neuron coupled to a tonically firing neuron by a spiking
synapse with different facilitation values.
Much of this behavior can be compared to circuit components. A parallel resistor-
capacitor circuit performs leaky integration when a voltage is applied across it. A spike is
similar to a transistor becoming excited when the base-emitter voltage surpasses a
threshold. These analogies may be useful to engineers in building models.
Chapter 3.1.3.2 – Nonspiking Integrate and Fire Model
The nonspiking IF model simulates single nonspiking neurons or the sum activity
of populations of spiking neurons. This model was used for two reasons. First, many
neurons in the stick insect motor control system are nonspiking, specifically those that
make up CPGs, interneurons that regulate muscle activity, and interneurons from sensory
inputs (Büschges, Kittmann, and Schmitz 1994; Büschges 1995; Büschges et al. 2004).
Second, the bandwidth of a spiking neuron is fundamentally limited by its maximum
firing rate (Trappenberg 2009). Single spiking neurons that encode sensory information
can only update other parts of the system at its maximum firing rate or lower, which is
lower than that at which their dynamics are simulated. Using multiple redundant spiking
neurons would solve this problem, but this would slow simulation time. Nonspiking
neuron models can communicate at the same rate at which they are simulated, meaning
that more precise timing and better coordination can be obtained.
40
The nonspiking neuron model is a leaky integrator that integrates its input
currents according to its membrane resistance and capacitance. This model can receive
inputs from synapses ( or other stimuli ( . It can also optionally include calcium
channels, which add another current term to be integrated. In addition, two additional
state variables must also be solved to calculate the calcium current. This current is due to
a gated conductance between the calcium voltage (200 mV) and the resting voltage of the
neuron. The dynamical equations are:
[( ( ]
(
( √ (
(
( √ (
where
( (
)
and
(
(
and are voltage gated variables. They cause overshoot in the neuron’s membrane
voltage when stimulated by an external current. Figure 6 shows the step response of the
nonspiking neuron with and without the calcium currents.
41
Figure 6 – Plots that show the nonspiking neuron’s step response
without calcium currents (top), and with calcium currents during
activation (bottom) and deactivation (bottom). All stimuli (green)
have a magnitude of 10 nA.
The nonspiking
model communicates with
other neurons via
nonspiking synapses. The
amount of current injected
into the postsynaptic neuron
( is determined by the
conductance between the
postsynaptic neuron’s
membrane voltage (
and the voltage of the
synapse ( . The
synapse’s voltage is a static
value. The conductance of
the synapse ( is
proportional to the
presynaptic ( voltage’s
distance between a low and
high threshold ( and
, respectively). This
can be mathematically
described as:
42
( )
{
This is much simpler than the spiking synapse model because it does not record
presynaptic history. However, it is less flexible because it cannot facilitate. The synaptic
strength cannot decay over time, meaning that phenomena that rely on synaptic
facilitation cannot be replicated with this model. A decaying nonspiking synapse was
written for use in the robotic leg controller:
( )
{
( ( ( ) ( (
The synapse conductance decays with a time constant of after the presynaptic neuron
crosses from below to above the conductance threshold of the synapse. This is more
computationally efficient than using the spiking neurons with facilitating synapses, which
must keep track of presynaptic spikes.
43
Chapter 4– Robust Robotic Stepping
SimROACH uses networks of these neuron and synapse models to excite muscles
and generate movement. Each leg possesses its own distributed control network, and each
uses the same types of sensory information (loading and joint angles) and actuators
(simulated muscles). However, they are structured differently to suit the role of each leg,
so they exhibit different motions. Each leg has a unique central pattern generator (CPG)
governing each joint. These CPGs are only coupled through sensory signals, not direct
connections among them. They act as gating clocks, changing the gain on muscular
positional control units in an oscillatory fashion, producing periodic motion. Since the
CPGs do not directly stimulate the motor neurons, the timing of the CPGs or the range of
motion of the joints can be altered while affecting the other minimally. Comparisons with
modified versions of SimROACH show that sensory coupled, dynamical CPGs make
stepping more robust to perturbations than alternatives.
Chapter 4.1 – Stepping Rules
Locomotion and the sensory cues that coordinate stepping have been the focus of
research for some time (Bucher et al. 2003; Akay et al. 2001; Ridgel et al. 1999; Zill,
Schmitz, and Büschges 2004; Zill, Keller, and Duke 2009; Zill et al. 2011). By
examining which muscles are activated or deactivated when certain sensors are excited,
sets of cause and effect rules have been uncovered and successfully modeled for stepping
in the middle leg of the stick insect (Akay et al. 2004; Ekeberg, Blümel, and Büschges
2004). These rules are typically described as reflexes, although the actual interactions are
more complicated.
44
These rules have been adapted for use in robotics, most notably in LegConNet
(Rutter et al. 2011). SimROACH uses the sensory pathways outlined in LegConNet, but
replaces the actuation of the ThC1 joint during walking with TrF actuation because of
data presented in (Bender, Simpson, and Ritzmann 2010). In LegConNet, the ThC1 joint
loaded and unloaded the leg while the CTr joint provided thrust (Rutter et al. 2011). In
SimROACH the CTr and TrF joints fill slightly different roles at different parts of the
stepping phase. Extending the CTr joint loads the leg, after which the TrF flexes (lowers).
At the end of stance the TrF extends to raise the tarsus, unloading the leg and causing the
CTr to flex in the return stroke. SimROACH actuates both extension and flexion of the
TrF joint although in Blaberus, the TrF can only be actively raised by the reductor
femoris and passively lowered by tendons (Carbonell 1947). Despite this difference,
SimROACH’s motion is more like that of a cockroach than with LegConNet because it
actuates the same joints as
Blaberus with similar timing.
The kinematics of
SimROACH are directly
compared to Blaberus in
Chapter 4.4.2 – Comparison to
Blaberus.
SimROACH’s stepping rules, adapted from LegConNet, can be seen in Figure 7.
Starting with the leg in swing, the CTr joint extends, loading the leg. This causes the FTi
joint to extend. At the same time, the TrF flexes. Once the CTr or FTi joint have reached
a critical angle, the TrF joint is extended to reduce the load on the leg. When the load is
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Flex -> Extend
TrF: Extend -> Flex
Fully EXT/Fully DEP TrF: Flex -> Extend
Leg load decreasing CTr: Extend -> Flex
Leg unloaded FTi: Extend -> Flex
Fully FLX CTr: Flex -> Extend
Figure 7 – Table of sensory triggers used to generate forward
walking in the middle leg.
45
decreasing, the CTr joint flexes, fully unloading the leg. At this point the FTi joint flexes,
which triggers CTr depression to reload the leg.
These rules were adapted to the other legs, which perform different tasks and
exhibit different motion than the middle leg. The hind leg follows the same stepping rules
as the middle leg while walking forward, but with joints sweeping different angles. The
front leg exhibits a reaching type motion that includes actuation of the complicated three
degree of freedom ThC joint as well as the CTr and FTi joints. The TrF joint is fused in
the front leg of Blaberus (Bender, Simpson, and Ritzmann 2010) and is locked in
SimROACH.
The front leg of the cockroach
has not been studied to
produce a set of stepping
rules. Therefore the rules in
the middle leg were modified
to produce the coordination
seen in the front leg of the
cockroach. The primary difference is that the ThC1 joint is used to generate thrust and
support, much like the CTr joint in the middle leg. Other differences are that the FTi joint
extends when unloaded instead of flexing as in the middle leg, and the ThC3 joint, not the
TrF, reduces the load to signal for swing to begin. These rules are summarized in Figure
8. This list of rules produces stepping that is coordinated and qualitatively resembles that
observed in Blaberus.
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Extend -> Flex
ThC1: Flex -> Extend
ThC3: Extend -> Flex
Fully FLX/Fully DEP CTr: Extend -> Flex
Leg load decreasing ThC3: Flex -> Extend
Leg unloaded FTi: FLX -> EXT
ThC1: Extend -> Flex
Fully EXT CTr: Flex -> Extend
Figure 8 – Table of sensory triggers used to generate forward
walking in the front leg.
46
Chapter 4.2 – Implementation of Stepping Rules
The stepping rules for each leg were implemented via a simulated network of
neural population models. Even though each leg’s network is different, the structure is
the same in each one. At the “top” of the structure are various sensory influences, such as
angle states and loading information. This information then passes through a network of
interneurons that are excited or inhibited based on which gait is active. These neurons
synapse onto interneurons in the CPG, and their excitation strengthens the reciprocal
inhibition of the half-centers. The CPGs then change the gain on muscle positional
controls, causing one antagonistic muscle or the other to contract. This causes motion
that affects the sensory neurons.
Chapter 4.2.1 – Sensory Information
The sensory information used in SimROACH is not identical to that in the animal,
but was designed to be analogous and provide the same types of information. The
primary sensory influences used by these stepping rules are joint angles and loading
information (Akay et al. 2004; Ekeberg, Blümel, and Büschges 2004). In stick insects
information about FTi flexion is provided by the femoral chordotonal organ (fCO), which
codes for organ length and velocity. Experiments in which the organ was directly
mechanically stimulated show that fCO output affects motor activity around the CTr joint
much like a switch, producing the strongest output at the extreme angles of normal FTi
motion during walking (Bucher et al. 2003). This suggests that the fCO’s effect on
interjoint coordination is step-like, even if the organ does code for stretch over the entire
range of its motion.
Instead of simulating the fCO, SimROACH uses the actual joint angle as
measured by the physics engine. The joint angle is transduced to a current by a linear
47
transfer function, which is injected into a neuron to code for FTi position. For
consistency, the slope of all joint angle transductions is always +/- 10 nA per radian. The
line is then shifted up or down according to the desired mean angle. Figure 9 shows the
angle, injected current, and neuron voltage for FTi extension of the middle leg when all
other joints are locked and the FTi joint was given an arbitrary mechanical stimulus. The
time constant of the position neurons was short (5 ms) so the integration lag between
angle state and the neuron’s state was negligible, as seen in the plots.
To obtain performance like that observed in (Bucher et al. 2003), the FTi position
neurons communicated with the rest of the circuit through synapses with relatively high
thresholds for conductance (-47 mV). This produced behavior like that seen in stick
insects preparations in which stretching the fCO beyond a certain point changed the
flexion or extension state of the CTr joint. Plots from the model demonstrating this can be
seen in Figure 10.
Figure 9 – Plots showing the measured angle (top), transduced current
(middle), and resulting voltage of a neuron coding for the joint’s
rotation (bottom). The gray lines show that the integration lag between
the current and the neuron’s voltage is virtually nonexistent.
48
In addition to FTi
excursion, stick insects and
cockroaches use loading
information to coordinate
stepping (Akay et al. 2004; Zill,
Schmitz, and Büschges 2004).
Loading information breaks
stepping up into two basic
motions: stance, which supports
and propels the animal while in
contact with the ground, and swing, when the leg is being returned to an anterior position.
Insects distinguish these states by measuring the strain of their legs through campaniform
sensilla (CS). CS detect load amplitude, rate of load, whether the rate is positive or
negative, and the lack of load (Zill, Schmitz, and Büschges 2004). SimROACH can
detect the same signals.
The population of CS on the trochanter are the most important for maintaining
coordinated stepping (Akay et al. 2004). Because of modeling constraints, SimROACH
does not calculate the strain on the trochanter, but instead uses the magnitude of the
normal force acting on the tarsus to detect load. This signal is transduced to a current in a
linear fashion and injected into the Load neuron, shown in Figure 11 (Neuron A). This
neuron is a spiking neuron with a low spiking threshold (2 mV above rest) such that its
firing frequency is a function of the load. In addition, its spiking threshold
accommodates, making the firing frequency also depend on the rate of load increase. The
Figure 10 – Plots showing the measured angle (top) and the
current injected into the rest of the system to signal that an
extreme position has been reached (bottom). There is no output
until the joint reaches a certain limit, at which point it rapidly
increases. The gray lines show what FTi angles signal to the rest
of the network.
49
Load neuron strongly inhibits the Unload neuron (Figure 11 B), which has a tonic drive
to make it fire when Load becomes less active at the end of stance. The Unload neuron
excites the Load Decreasing neuron (Figure 11 C) with a facilitating synapse that decays
over time, causing it to fire briefly as the load decreases to nothing.
Figure 11 compares data from SimROACH to neural recordings from
cockroaches. SimROACH can produce signals that encode leg loading similarly to
cockroaches despite not measuring load in the same way. The robotic implementation of
a single leg from SimROACH uses a strain gage on the trochanter to detect load, much
Figure 11 – (Top) Unpublished results from the Zill lab showing how some populations in the cockroach
respond to increasing load while others respond to decreasing load. (Bottom) Picture of network that
processing loading information. Neuron A turns the signal D, the magnitude of the load on the foot, into a
firing frequency (E.). Neuron B is inhibited by neuron A, and will fire when the load goes away. Neuron C
is stimulated by Neuron B with a decaying synapse, so its activity decays rapidly when the load ends (F.).
50
like CS. The robot was able to walk normally with this substitution, suggesting that
detecting load from the tarsus is an acceptable substitution to help maintain stepping
coordination. However, insects use CS signals from other locations on the leg to monitor
muscle tension and leg orientation (Akay et al. 2004; Noah et al. 2004; Watson,
Ritzmann, and Pollack 2002), something that could benefit SimROACH in the future.
Chapter 4.2.2 – Sensory Interneurons and Reflex Reversal
Sensory information in SimROACH affects CPG timing through a layer of
interneurons. Different gaits in insects are generated by changing how only one or two
sensory inputs are processed, changing the order in which joints are flexed or extended
(Akay et al. 2007). SimROACH’s primary goal is to produce flexible robotic behavior in
this fashion, so this effect was replicated. Researchers in stick insect neurobiology have
hypothesized that such changes could occur by changing the weight of parallel sensory
pathways via descending commands (Akay et al. 2007). Such a method has been
successfully implemented in both computational models (Daun-Gruhn 2010) and
biologically inspired robots (Rutter et al.
2011). The specific rules that change in
SimROACH’s different behaviors will
be discussed in Chapter 5 – Smooth Low
Level Transitions, but the mechanism
that accomplishes this will be described
here.
SimROACH reverses reflexes by
changing the excitation of interneurons
Figure 12 – Schematic of how a reflex reversal can be
executed in this model. The Gait neuron can affect
interneurons that relay sensory information, changing
which neurons are affected by which sensors.
51
that conduct a sensory signal to a CPG or multiple CPGs. Figure 12 shows a simple
circuit that illustrates this mechanism. Path A and Path B’s membrane voltages will
reflect the signal produced by Sensory Signal. When the Gait neuron is not excited, it
does not affect these interneurons. Synapse 3 between Path A and the Flex neuron is
configured such that in this case Path A relays information from Sensory Signal to the
Flex neuron. Synapse 4 between Path B and the Extend neuron has a high conduction
threshold, such that the voltage on Path B cannot affect the Extend neuron. When the
Gait neuron is excited, however, the voltage of Path A is suppressed such that its signal
remains below the conduction threshold for synapse 3. Conversely Path B is excited up to
the conduction threshold of synapse 4, enabling it to conduct information from Sensory
Signal to the Extend neuron.
It is desirable to make smooth transitions between gaits rather than
discontinuously changing behavior. The time constant of Gait neurons in this model,
whether at the low or intermediate level, was 500 ms unless otherwise noted. This
corresponds to reaching 99.7% of full excitation after 1500 ms of a step input, consistent
with the observation that cockroaches change between forward walking and turning over
the course of about 1500 ms (Brown 2011).
Chapter 4.2.3 – Central Pattern Generators
52
The CPG model used in
SimROACH is a nonspiking half-center
oscillator. It is composed of four neurons,
two that generate the rhythm and two that
serve as interneurons between the rhythm
generators. A schematic of this structure
can be seen in Figure 13. Neurons 2 and 4
are the interneurons, and have properties
similar to most others in this simulation.
Neurons 1 and 3 differ in that they utilize the optional calcium channels, which provide
additional dynamics that lead to positive feedback in the membrane’s response to input
current as discussed in Chapter 3.1.3.2 – Nonspiking Integrate and Fire Model. Among
all four neurons, each CPG has eight state variables: two neurons each with a membrane
voltage and two each with a membrane voltage, a calcium channel activation state, and a
calcium channel deactivation state.
Figure 13 – Schematic of a CPG used in this model.
Neurons 1 and 3 are the half-centers, communicating
through interneurons 2 and 4.
Figure 14 – Voltage of one half-center of a CPG during oscillation. The oscillation reaches steady state
after about 1 second.
53
The CPGs naturally oscillate without input, forming a limit cycle with all eight
state variables. An example of the output of one of the half-centers is shown in Figure 14.
Numerical simulations with XPP show that this limit cycle possesses exactly one
Figure 15 – Voltage of one half-center of a CPG during normal activity (top) and when the interneurons are
inhibited by a current of -1.25 nA.
54
equilibrium point, which has four real negative, two negative complex, and two positive
complex eigenvalues. The strongest set is , meaning the equilibrium point
is unstable. The observed oscillation tells us that a limit cycle forms, so this is the only
Figure 16 – Voltage of one half-center of a CPG when the presynaptic neuron is strongly hyperpolarized at
different points in the phase. A strong enough stimulus will reset the phase of the CPG at any point of the
phase. The bars along the bottom show the bursting period before the stimulus was applied. A red circle is
drawn around the perturbation, after which a normal looking period of activity is observed.
55
stable configuration of the system with this set of parameters and inputs.
This CPG model replicates experiments performed on physiological CPGs in
organisms. One of the defining characteristics of a CPG is that a strong hyperpolarization
of one neuron will cause the oscillation to reset, with the inhibited neuron firing a full
burst when released from inhibition. Experiments with this CPG model show that
strongly hyperpolarizing one of the interneurons between the half-centers resets the
oscillation regardless of the phase of stimulation, consistent with results from the
heartbeat of the leech (Arbas and Calabrese 1987) and motor systems of stick insects
(Büschges 1995). Figure 16 shows the voltage plots of one half-center when the
presynaptic neuron is excited, hyperpolarizing the half-center (5 nA pulse 100 ms long).
The hyperpolarization swiftly ends the current phase of oscillation and causes an intact,
full period of activity immediately following, shifting the phase of oscillation.
This CPG model is also capable of producing a wide range of frequencies.
Without external input, the CPG will oscillate at a frequency determined by the rates of
calcium channel activation and deactivation in neurons 1 and 3. Inhibiting interneurons 2
and 4 weakens the inhibition between neurons 1 and 3, decreasing the CPG’s oscillation
rate as seen in stick insects (Büschges et al. 2004). Figure 15 shows output of the CPG
both without external input to the interneurons and with -1.34 nA tonic drive. Such
stimulus reduces the period by 80%. This means this CPG can potentially be used for a
variety of gait speeds, although muscle properties and sensory information also influence
the rate of stepping (Pearson 1993; Mackay-lyons 2002).
Chapter 4.2.4 – Muscle Control Units
SimROACH uses muscles as actuators. Muscles were chosen for two main
reasons: compliance and biological consistency. Muscles are compliant actuators, and can
56
be modeled as a system of springs and dampers as described in Chapter 3.1.2 – Muscle
Model. Complaint actuators interest engineers because they passively reject perturbation
and require less precise control (Jindrich and Full 2002; Loeb, Brown, and Cheng 1999;
Kingsley, Quinn, and Ritzmann 2006). Not needing to respond to every single disruption
while stepping reduces both computational resources needed for control and energy
consumption during stepping. The second reason SimROACH uses muscles is to more
readily couple the neural control system with the motor output; they have evolved to
work together.
CPGs in stick insects do not excite muscles, but rather inhibit them from an
excited state (Büschges et al. 2004). It is also known that sensory influences can affect
muscle activation without affecting CPG timing (Zill, Schmitz, and Büschges 2004; Akay
et al. 2001; Akay et al. 2004; Büschges 1995) . In addition, nonspiking neurons modulate
motor output based on CPG activity and sensory feedback (Büschges 1995). SimROACH
generates controlled muscle tensions through an engineered system based on what is
known about insect muscle control. Sasha Zill guided the development of this system.
Each joint has one CPG that alternates between two antagonistic muscles. Each
joint also has two equilibrium positions, fully flexed and fully extended, that are set as
static values. Positional control systems built from neurons exist for each position, and
both are active at all times. Damping is present in the muscles, so the system can be
considered positional-derivative (PD) control. The CPGs modify the gain on each,
causing one equilibrium position to be more attractive than the other. The error between
the desired angle and the current angle is calculated for both the fully flexed and the fully
extended position. These signals, one for each muscle, are then gated by the appropriate
57
CPG half-center. These positional errors are used to generate a force, which can lead to
complicated dynamics. To better understand these dynamics, an abstracted version of this
system was analyzed using XPP.
The muscle control unit was initially simulated without CPGs by the following
dynamical system:
( (
Where x is the position (angle), x1 and x2 are the desired flexion and extension
equilibrium points, k1 and k2 are gain values on each positional error, and b is damping
due to muscle dynamics. In this form, the system will move toward whichever term
provides the highest drive. For instance, if x1 equals -x2 but k1 is five times k2, the
system’s attraction to x1 will be five times as high and the position will settle at
.
An example of such behavior can be seen in Figure 17(B).
The effect of CPGs can be examined by adding a sinusoidal component to the
gain terms for each equilibrium point. If the gains are allowed to oscillate between 0 and
1 and are 180 degrees out of phase, the new dynamical system looks like:
( ( ) (
( ( ) (
58
where omega is the frequency of the CPG. Using XPP, parameters in the system were
varied and behavior changes were documented. Figure 17 shows some of these results.
Initially, omega was set to 1, k1 and k2 were set to 1, and x1 and x2 were set to 1 and -1,
respectively. In this configuration, the system oscillates at the desired frequency and
A
B
C
D
E
F
Figure 17 – Behavior of the abstracted muscle pair system. If both stiffnesses (1,1) and set points (1,-1) are
identical and the sin terms are removed, any initial condition will drive the system toward an angle between
the two set points (A). Changing the stiffness of one set point (k1=5*k2), the system will move toward that
set point (B). If the sin terms are included, the system can be tuned to oscillate with the desired frequency
and amplitude (C). If the frequency is changed, the amplitude is decreased, since this system is a filter (D).
The desired amplitude can be regained by increasing the stiffness of both set points (E). This is not
something the current model is capable of. Keeping the original stiffnesses and instead increasing the set
points will produce qualitatively similar behavior (F). This is the approach that the current model uses to
increase the stiffness of a joint.
59
amplitude (Figure 17 (C)). If omega is
increased to 2, however, the system no
longer achieves the desired range of
motion due to the transmissibility of the
system (Figure 17 (D)). Increasing the
stiffness of each positional controller, k,
helps regain the desired amplitude by
changing the band of the pass filter
(Figure 17 (E)). Achieving this in
SimROACH would require increasing
the stiffness of the series element
springs, something that the model
cannot do. Cockroaches accelerate
running speed by activating fast muscle
fibers in each joint as it changes
direction (Watson and Ritzmann 1998).
SimROACH could be modified to do
this, but it does not possess different
muscle fiber types. Instead the
equilibrium points are set further away
from one another, producing larger
errors and therefore larger forces (Figure
17 (F)). SimROACH uses this method to
Figure 18 – Plots of joint angles (blue) and extreme
position neuron voltages (green). Note that flexion can
be changed independently of extension (top) and vice
versa (middle). They can also be changed together to
change the mean angle (bottom).
60
increase reaction forces during stance and change kinematics during different gaits.
This control method was tested in SimROACH in a reduced preparation. All
joints on the mesothoracic leg were locked except the FTi joint. All sensory influences
were removed from the FTi CPG, allowing it to oscillate at the CPG’s natural rate. The
desired position was then changed as the joint oscillated, changing the range of motion.
Plots in Figure 18 show these changes. As long as the muscle properties are configured
properly, one can change the desired flexed position and not affect the range of extension
(Figure 18 (A)), and vice versa (Figure 18 (B)). In addition to increasing or reducing the
range of motion of a joint, the mean position can also be shifted (Figure 18 (C)). These
effects have been observed in some
cockroach gait changes (Brown 2011).
The structure of the muscle control
subsystem is shown in Figure 19. This
particular image is from the CTr joint of
the middle leg, so it can depress (DEP) or
levate (LEV). In the yellow box is the
feedback control for the DEP muscle. The
voltage of the DEP FB (depressor
feedback) neuron codes for the angle of
the CTr joint as discussed in Chapter 4.2.1
– Sensory Information. The Extreme POS
LEV (extreme position levation) and
Extreme POS DEP (extreme position
Figure 19 – Schematic of the CPG and muscle
control unit for the CTr joint. The CPG (red) only
inhibits the Inter Pos neurons, which are interneurons
between the error feedback control for each muscle
and its motor neuron.
61
depression) neurons code for the maximum angle for each direction. These can be
changed by sensory influences or descending commands. The eLEV (levation error) and
eDEP (depressor error) neurons are comparators between the actual angle and the
extreme angles. These stimulate interneurons Inter Pos DEP (interneuron position
depressor) and Inter Pos LEV (interneuron position levator), which receive inhibitory
input from the CPG. The CPG reduces gain of the positional control units in a periodic
manner, generating oscillatory motion. A bias was added to the error of both sides (+10
mV) to maintain a baseline level of stiffness.
Chapter 4.3 – Networks and Their Function
Three networks were developed according to the structure described in Chapter
4.2 – Implementation of Stepping Rules, one for each leg of the cockroach. The front leg
is capable of the most agile motion including reaching motions by actuating its proximal
joints, and brakes the animal’s forward motion. The middle legs provide support and
braking during the first half of stance and thrust in the second half. The hind legs provide
support and most of the thrust during walking, and follow the same stepping rules as the
middle leg (Full, Blickhan, and Ting 1991).
Each leg has five actuated degrees of freedom. The rear two legs actuate the
ThC2, ThC1, CTr, TrF, and FTi joints, while the front leg actuates the ThC3, ThC2,
ThC1, CTr, and FTi joints. Figure 2 shows a photograph of a cockroach leg, as well as
screenshots of each leg from the simulator with the degrees of freedom labeled.
Chapter 4.3.1 – Middle Leg Network
62
The middle leg control network is shown in Figure 20. The neurons are color
coded and arranged in a hierarchical fashion to make their purposes clear. The light blue
neurons along the top are sensory neurons. These include information about loading and
proprioception. The details can be found in Chapter 4.2.1 – Sensory Information. If this
network could not produce different gaits, these neurons would directly synapse to the
Figure 20 – Control network for the middle leg of the cockroach with no particular gait active (top) and
with the forward walking gait active (bottom). The inactive pathways have reduced fill.
63
top layer of the CPGs, shown in red. Instead these neurons synapse onto the dark blue
sensory interneurons, which can be excited or inhibited by the green gait neurons along
the sides. These neurons produce reflex reversals. The details are explained in Chapter
4.2.2 – Sensory Interneurons and Reflex Reversal. These neurons influence the
interneurons of the CPGs (top red), which control the strength of inhibition, and thus
frequency of the CPG half-centers (bottom red). The CPGs change the gain on muscle
control units as discussed in Chapter 4.2.4 – Muscle Control Units.
This network can be overwhelming to look at, so Figure 20 also shows the
network with reduced fill on the neurons that are not used for walking. The rules that this
network encapsulates are listed in Figure 7. In addition, the properties of every neuron
and synapse are listed in Appendix A – Network Topologies. The network was
constructed from neurons and synapses with as few unique parameter sets as possible to
simplify recreation on board a robot. More attention to detail might improve
performance, something discussed in Chapter 7 – Conclusions and Future Work.
Chapter 4.3.2 – Front Leg Network
The front leg network, shown in Figure 21, is noticeably larger than the middle
leg network because of the larger group of dark blue sensory interneurons. Not only does
the front leg utilize more joints than the other legs in most gaits, but it is also changes its
behavior the most between them. The stepping rules used for forward walking are shown
in Figure 8, and Figure 21 includes a picture of the network highlighting pathways that
are active during forward walking. The color scheme of the neurons is the same as listed
in Chapter 4.3.1 – Middle Leg Network, and a list of every neuron and synapse and its
properties is provided in Appendix A – Network Topologies.
64
Chapter 4.3.3 – Hind Leg Network
The network that generates stepping in the hind leg is shown in Figure 22. As
noted previously, the hind leg does not change its behavior when Blaberus turns, so this
network contains no dark blue sensory interneurons for reversing reflexes. As with the
Figure 21 – Control network for the front leg of the cockroach with no particular gait active (top) and with
the forward walking gait active (bottom). The inactive pathways have reduced fill.
65
other legs, a table of all of the properties of the neurons and synapse is available in
Appendix A – Network Topologies.
This network differs from the others in that the CTr and FTi joints exchange
proprioceptive information in order to coordinate their motion. They should extend and
flex at the same angles, so the difference between the angles is computed to determine
how much the CTr joint is over flexed or over extended. The error is then used to
stimulate the motor neuron for flexion or extension, respectively, of the FTi joint. This
method was developed because there is no evidence that CPGs can influence one another
directly to maintain coordination (Büschges, Schmitz, and Bässler 1995). However it is
not perfect, and could be improved by using an actual controller to maintain this relative
angle rather than a simple comparator.
An engineering solution would be to coordinate the joints by sharing the same
CPG. However such a decision would directly contradict the secondary goal of making as
Figure 22 – Control network for the hind leg of the cockroach. This network is much smaller than the
others because no reflex reversals take place.
66
accurate a biological model as possible, and could potentially limit SimROACH’s
behavioral flexibility as more functionality is added in the future.
Chapter 4.4 – Stepping Results
The performance of such a biomimetic system can be judged in two ways:
similarity to the animal and general engineering effectiveness. The intent of SimROACH
is to make robots walk more robustly, so the primary metric was to produce effective
stepping. However, cockroaches are some of the most agile hexapods and a robot that
could move like a cockroach would be extremely effective. Further, direct comparison
with cockroach movements is instructive because data for that model organism is
available. SimROACH’s motion while walking is not identical to that of the animal, but
is similar enough to draw comparisons. In addition, some results from biology can be
replicated by measuring neural or muscle activity.
Chapter 4.4.1 – Stepping Robustness
The goal of this section is to show that the stepping this system produces is robust
when confronted with various challenges that a robot might face. All experiments were
performed in simulation. The first experiments show that it can adjust to topographical
changes, maintaining coordination as the body changes elevation or stepping in a hole.
The second group of experiments shows how it can adjust to changes in its own form,
such as extra load from an impediment or loss of communication from a sensor. In each
of these cases coordination is maintained even if the walking pattern changes.
Chapter 4.4.1.1 – Elevation Change Experiments
Experiments were performed in which the simulated middle leg was attached to a
test stand and made to walk on a frictionless surface, which models the robot experiment
described later. The stand allowed the leg to lift the attachment point in stance, but had a
67
minimum height for the leg to simulate the support of the other leg. Two different tests
were conducted. During the first, the attachment point of the leg was raised slowly as it
walked, forcing the leg to reach lower and lower to make contact with the ground. The
stepping rules state that the leg will depress the CTr joint until loading occurs and the FTi
joint extends, so coordination is maintained as long as the leg can reach the ground. It
could step in a coordinated fashion until lifted to a height of 1.7 cm, 170% of the height
of the ThC1 joint. At this height it could not reach low enough to make ground contact
and coordination disappeared.
Similar experiments were performed in which the leg’s minimum height was
increased for only one stance phase, simulating a step in a hole. This is essentially no
different from gradually changing the height as noted previously, except that the change
is much faster. The middle leg could successfully step through a hole that was .73 cm
deep, which is 56% of its standing height. After returning to normal stepping height, it
continued to walk without disruption.
Experiments were performed in which the stance phase was cut short by raising
the leg during a step. Manually lifting the leg unloaded it, reducing propulsive forces in
the muscles and causing the FTi joint to flex, which is a part of swing. Experiments in
which swing was interrupted were also performed. The loading information caused the
CPGs to transition to their stance states and then continue to step normally. These
experiments show how SimROACH can adapt its stepping to unexpected obstructions.
Chapter 4.4.1.2 – Body Manipulation Experiments
The CPGs in SimROACH are intended to maintain stepping rhythm when the
stepping motion is changed and normal sensory thresholds are not crossed. In order to
show that CPGs help the leg continue stepping when the dynamics of the body change,
68
two versions of the middle leg were developed, one with CPGs and one without. That
without CPGs needed sensory information to maintain rhythm, so restricted motion could
halt stepping. Experiments were performed in which the foot segment’s density was
increased, mimicking situations when the environment might limit the leg’s range of
motion (e.g. stepping through mud, dragging along debris, etc.). In these experiments, the
middle leg was attached to a simulated stand as in the previous section. Under normal
conditions, both the leg with CPGs and that without CPGs were able to generate stepping
motion. When the weight of the foot was increased, the simulation without CPGs ceased
stepping because its range of motion was limited. This prevented it from reaching its
normal sensory thresholds, and the reflex cascade halted. The model with CPGs, however,
continued stepping despite the limited range of motion. Figure 23 shows the kinematic
output of the leg during this trial. The version with CPG model steps with high frequency
oscillation due to the extra mass, but maintains rhythm despite this.
69
CPGs cause stepping to
continue in the absence of one
sensory cue. This experiment
simulates scenarios in which the
robot’s sensors are damaged, are
removed, or malfunction in the field.
This level of robustness would benefit
a robot in a disaster zone or other
dangerous environments. In these
experiments, the middle leg was
attached to a simulated stand as
above. Both the leg with CPGs and
without was able to walk forward
with normal sensory input. However,
when the positional signal from the
FTi joint was eliminated, the version
without CPGs stopped walking. This
is because that feedback was
necessary to drive the next joint
transition in the reflex cascade. The
model with CPGs, despite changed
kinematics, was still able to move the
joint in time with the other joints and
Figure 23 – Three plots showing kinematics during
walking in a middle leg without CPGs and under normal
load (top), without CPGs during weighted walking
(middle), and with CPGs during weighted walking
(bottom). The leg is able to walk under normal conditions,
but adding extra weight stops the reflex cascade. Adding
CPGs to the model restores rhythmic behavior. The extra
inertia causes high frequency noise in the kinematics that
would otherwise be absent. Gray shading indicates stance.
70
continue the walking motion.
Plots of the kinematics from
these trials are shown in Figure 24.
Without CPGs, the simulation took a
single step and then stopped. No FTi
input means the leg would not extend
its CTr joint, so the leg was artificially
loaded around 2 s and 2.5 s, after
which the leg took a step. However, it
could not sustain the reflex cascade
indefinitely without FTi information.
This experiment highlights how CPGs
can improve the rhythmicity of
stepping, even when parts of the
system are not intact.
Chapter 4.4.2 – Comparison to Blaberus
Kinematic data were collected from each leg during walking with the tripod gait.
These data will be compared to joint angles on Blaberus during walking recorded by
Amy Brown in the Ritzmann lab.
Figure 24 – Plots showing kinematics during walking for a
middle leg without feedback from one joint in a model
without CPGs (top) and a model with CPGs (bottom). A
CPG at every joint reduces the robot’s reliance on sensory
information in case of a malfunction.
71
The prothoracic leg possesses a complex three degree of freedom joint connecting
its thorax and coxa. According to Brown’s data the excursion each one makes during
walking varies a fair amount, so general trends were used to produce SimROACH.
During walking, the ThC2 joint is relatively inactive, and the ThC1 and ThC3 joints
provide thrust and unload the leg. These two joints are highly active during walking,
exhibiting average joint excursions of 0.471 radians and 0.596 radians, respectively.
Figure 25 shows the kinematics the front legs of Blaberus and SimROACH while
walking. In the cockroach the ThC3 and FTi joints extend and flex nearly in phase,
something that SimROACH mimics. In addition, both flex the CTr joint during the stance
phase, although SimROACH flexes at the end of phase rather than gradually throughout.
SimROACH does not actuate the ThC1 and ThC2 joints properly. This is largely due to
Figure 25 – Joint angles of the front leg during tripod walking. The kinematics of the animal (left) were
recorded by Brown on an oiled plate. SimROACH’s kinematics (right) are provided for comparison. The
vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated
by gray shading.
72
the difficulty in tuning muscles to produce the desired range of motion. This issue is
discussed in Chapter 7 – Conclusions and Future Work.
Figure 26 shows kinematic data from the middle leg of Blaberus and
SimROACH. The middle leg matches biological data better than the front leg, producing
similar ranges of motion and phase relationships. The CTr and FTi joints extend during
stance, and the TrF joint extends during swing to position the leg for loading. In addition
to kinematics, loading information, muscle activations, and the order in which they occur
are similar to data recorded in the America cockroach Periplaneta americana. Figure 11
(See Chapter 4.2.1 – Sensory Information) shows the response of various neural
populations in the cockroach to campaniform sensilum stimulation. As discussed in
Chapter 4.2.1 – Sensory Information, SimROACH’s load detection mimics that found in
the cockroach.
Figure 26 – Joint angles of the middle leg during tripod walking. The kinematics of the animal (left) were
recorded by Brown on an oiled plate. SimROACH’s kinematics (right) are provided for comparison. The
vertical axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated
by gray shading.
73
In addition to similar load processing, SimROACH activates leg muscles in the same
order as cockroaches. In the American cockroach, the CTr extensor is active before the
FTi extensor during stepping. SimROACH mimics this result because its stepping rules
state that CTr extension causes loading, which causes FTi extension. Recordings from the
cockroach and data from SimROACH are compared in Figure 27. This result suggests
that the stepping rules that end swing and initiate stance are biologically accurate.
The hind legs can be compared to data collected from Blaberus, shown in Figure
28. In the organism, the CTr and FTi joints are nearly locked in both phase and
amplitude, a feature that was attempted in this model. This locking produces long
propulsive strides. The mechanism that causes this is unknown, so SimROACH uses a
comparator between the CTr and FTi joints detailed in Chapter 4.3.3 – Hind Leg
Figure 28 – Joint angles of the hind leg during walking. The kinematics of the animal (left) were recorded
by Brown on an oiled plate. SimROACH’s kinematics (right) are provided for comparison. The vertical
axes are scaled to match biological data, and are the same in both graphs. Stance phase is indicated by gray
shading.
74
Network. Figure 28 shows that this system locks the phase but not the amplitude of CTr
and FTi motion.
Chapter 4.5 – Robotic Implementation
A robotic model of the cockroach middle leg was built by Matt Klein (Figure 30)
for experimentation on insect load sensing with Sasha Zill. The robot has five degrees of
freedom actuated by Dynamixel AX-12+ smart servos. Sensors include potentiometers at
each servo and strain gauges on the trochanter that mimic load sensors found on the
cockroach (Zill, Schmitz, and Büschges 2004; Zill et al. 2011). Neural simulation was
performed with LabVIEW (National Instruments, Austin, TX) and run on a laptop (2.0
Figure 29 – Plots comparing muscle activations with the onset of stance in Blaberus discoidalis (top) and
SimROACH (bottom). In both systems the CTr joint is depressed to cause stance, which causes the
extension of the FTi joint. The biological data was produced by the Zill lab. Stance is indicated in the
bottom plot by gray shading. Top figure used with permission from Sasha Zill.
75
GHz Intel Core2Duo). The laptop is
connected wirelessly to a NI
CompactRIO-9074 which handles all
communication with the servos and
sensors.
The robotic leg used the circuit
shown in Figure 20 to walk. Neurons not
used for forward walking were removed
for simplicity. This work proved the
concept of using simulated nervous
systems to control walking in a legged
robot. The network ended up possessing
22 neurons, including three CPGs. Only the three most important joints of the leg (FTi,
TrF, and CTr) were rhythmically actuated.
The muscle control units from SimROACH were adapted to the robot. Rather
than performing the extra calculations needed to simulate the full muscle control unit, the
entire system was abstracted. Each joint was assigned maximum and minimum joint
excursion values. Half-center voltages were compared, and the more excited half-center’s
associated equilibrium point was sent to the servo. The servos only updated once every
50 neural timesteps, making this abstraction necessary for smooth motion. Smoothness
was also increased by setting the servo compliance to its maximum value. This reduced
the amount of torque the servos applied for a given positional error, decreasing the
acceleration.
Figure 30 – Picture of the robotic leg used for
hardware testing (A). It manages input and output
through a NI CompactRIO (B) and outputs data to
LabView (C).
76
The robotic leg also differs from SimROACH in that its load detection is much
more like in the animal. The robot has strain gages on the trochanter and tibia oriented in
the same way as sets of campaniform sensilla in cockroaches. The readings from these
were converted to neural activity in the same way as in SimROACH, by turning the load
into a current to be injected into a neuron. In this implementation, only one strain gage on
the trochanter fed into the coordinating circuit because it is most important to
coordinating stepping (Akay et al. 2004). In the future, input from the others will be used
to modify simulated muscle
activity in other joints.
Kinematic output of
walking is shown in Figure 31.
The joint excursions are nearly
linear, which does not look
organic. Figure 31 also shows
CPG output, which is clearly
coordinated in the desired fashion,
with CTr extension loading the
leg, FTi extension signaling for
unloading, and TrF extension
signaling for full unloading.
All computation was
performed on a laptop for these
experiments, but an actual robot
Figure 31 – Joint Angles (top) and CPG activity (bottom) from
a walking trial performed with the robotic leg. Stance is
indicated by gray shading.
77
would need to perform calculations on board. A typical microcontroller would not be able
to simulate the neural system in real time. Field Programmable Gate Arrays (FPGA),
however, have been shown to perform this type of simulation in real time (Cheung et al.
2006). An FPGA uses a network of logic gates to physically create the circuit as
interpreted by a compiler. This means an FPGA implementation of this system would
actually build circuits that behaved like neurons, and then send information among them
to simulate their interactions. As a proof of concept, one CPG (four neurons) was written
to the FPGA built into the CompactRIO. The size of the network was limited by the
storage capacity of the FPGA used. This FPGA properly simulated the CPG very rapidly,
calculating a 1 ms integration step in only 9 µs. This is about twenty-five times faster
than the 230 µs run time for an identical network on the laptop. Furthermore, the
FPGA’s parallel structure means it can simulate a network of any size in the same
amount of time. Thus, as the network size increases, the FGPA’s performance margin
over traditional computers will also increase.
78
Chapter 5 – Smooth Low Level Transitions
SimROACH was designed not only to walk but also to smoothly and stably
transition between gaits. It can produce inside and outside turning motions with its front
and middle legs, allowing the body to turn while walking forward (Mu and Ritzmann
2005). This is accomplished by modifying the sensory pathways that couple the CPGs.
Some of these changes are based on (Rutter et al. 2011), as seen in Figure 32. Rules for
front leg gait changes were hypothesized based on kinematic data from (Brown 2011).
Results from LegConNet presented in (Rutter et al. 2011) show that it could only
change gait in a rapid, discontinuous way. If the command to turn were applied gradually,
stepping often stopped. Gradually changing gait should not only produce smoother
motion, but is also supported by work in cockroaches (Brown 2011). SimROACH
exhibits gradual, smooth, and stable gait transitions due to its use of naturally rhythmic
CPGs and gradual reflex reversals. Experiments show that removing the CPGs cause
these transitions to fail. When transition timing is set to match observations in Blaberus,
the presented model can change gait at any point in the stepping phase and smoothly
change kinematics to produce the desired behavioral change.
Figure 32 – Diagrams that explain LegConNet when producing forward (left) and inside turning forward
(right) behavior. Gait changes are generated by changing the connections and thresholds between sensory
influences and bistable “CPGs”. Taken with permission from (B L Rutter et al. 2011)
79
Chapter 5.1 – Implementing Behavior Changes via Reflex Reversals
As described in Chapter 4.2.3 – Central Pattern Generators, each joint of each leg
has its own CPG, which is not directly coupled to any other CPG. Instead, the CPGs are
coordinated through sensory influences. These sensory influences are then modified by
interneurons, allowing them to be reversed or rerouted by descending commands.
In SimROACH such modifications are essentially bias changes to the
interneurons, an idea familiar to perception networks. In typical neural nets, a neuron can
be biased to change the threshold above which it can communicate with other neurons.
Gait neurons (green in Figure 34) bias sensory interneurons in SimROACH to change
which sensory pathways are active. The neural structure is detailed in Chapter 4.2.2 –
Sensory Interneurons and Reflex Reversal. This biasing technique also has basis in
findings from (Hellekes et al. 2012). The authors suggest that descending commands
modify which sensory signals affect which joint. It is not known what part of the nervous
system causes these changes, that is, stimulates the green Gait neurons in the control
networks. Recordings in the central complex of cockroaches suggest that it may be the
source of such reflex reversals (Guo and Ritzmann 2012). Insects may not change
behaviors in exactly the same way that SimROACH does, but SimROACH is consistent
with what is known about turning behaviors.
In addition to reversing reflexes, CPGs must be able to be turned off (Daun-Gruhn
2010). It has been noted that different joints are actuated during different behaviors, so it
is necessary to turn them off in a reversible way. The CPG model used will cease
oscillating when sufficient inhibitory current is applied directly to the half-centers. When
this occurs, the muscle control units receive no modulation from the CPGs, and both
muscles are held taut as each control unit tries to reach its equilibrium point.
80
When examining biological joint angle data, a small range of motion for a joint in
the organism may correspond to oscillatory actuation from a CPG or the passive reaction
of the joint to the forces acting on the leg. The data compared to SimROACH come from
oil plate experiments with Blaberus, during which these effects are minimal. For
engineering simplicity, joints that are judged to traverse small angles are not actuated in
SimROACH.
Chapter 5.2 – Flexible Networks Capable of Changing Gait
As noted in Chapter 4.3 – Networks and Their Function, separate control
networks were developed for each leg. Each leg steps in a different manner, and gait
changes are caused by reflex reversals specific to each leg. The front and middle legs of
SimROACH can generate inside and outside turning behaviors, while the hind legs can
only walk forward. These are all that is necessary for producing turning (Mu and
Ritzmann 2005; Hellekes et al. 2012). The hind leg network, in its present state, cannot
change gait and therefore is not presented in this section.
81
Chapter 5.2.1 – Gait
Changes in the Middle Leg
The middle leg can turn
both inside and outside.
Outside stepping is
characterized by deactivation
of the TrF joint and activation
of the ThC2 to generate
outside pushing motion. In
addition, the CTr joint and FTi
joint end stance further
extended than usual (Brown
2011). Inside stepping is
characterized by reversing the
role of FTi flexion and
extension and activating the
ThC2 joint to extend the leg’s reach in swing. These rules are summarized in Figure 33,
and are implemented in the middle leg control network, shown in the appropriate forms
in Figure 34. Note that both features discussed previously, toggling CPGs and changing
kinematics, are used to produce these behavioral changes. In addition, these changes
occur when only one neuron in the circuit is stimulated, representing descending
commands’ influence on the low level circuit. As noted before, this neuron takes 1500 ms
to come to equilibrium, simulating the slow behavioral change observed in Blaberus.
MIDDLE LEG – INSIDE TURNING
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Extend -> Flex
TrF: Extend -> Flex
ThC2: Extend -> Flex
Fully FLX/Fully DEP TrF: Flex -> Extend
Leg load decreasing CTr: Extend -> Flex
Leg unloaded FTi: Flex -> Extend
ThC2: Flex -> Extend
Fully EXT CTr: Flex -> Extend
MIDDLE LEG – OUTSIDE TURNING
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Flex -> Extend
ThC2: Flex -> Extend
Leg load decreasing CTr: Extend -> Flex
Leg unloaded FTi: Extend -> Flex
ThC2: Extend -> Flex
Fully FLX CTr: Flex -> Extend
Figure 33 – Tables that show stepping rules for inside turning
(top) and outside turning (bottom) implemented in the middle leg
of this model. There is no one authoritative source for these
turning rules, but they are based on literature and hypothesized
transitions.
82
Evidence of these changes is most clearly seen in plots of CPG activity. Figure 35
shows the CPGs in the middle leg during the transitions from walking forward to turning
in either direction. During walking, CTr extension slightly leads FTi extension, and full
extension leads to CTr flexion followed by FTi flexion. Inside turning is more of a
Figure 34 – Control networks for inside turning (top) and outside turning (bottom) in the middle leg model.
The sensory pathways are highlighted to match the rules listed in Figure 33. The behavior changes are the
result of rerouting sensory information and turning CPGs off where necessary.
83
reaching and pulling motion, not a
pushing motion, so these joints switch:
CTr flexion causes FTi extension, which
causes CTr extension and FTi flexion.
This puts the tarsus further from the
thorax during swing and pulls inward
during stance. The ThC2 joint is also
activated during inside turning,
extending to protract the leg further
from the thorax during swing.
The middle leg can produce
outside stepping behavior by
deactivating the TrF joint and activating
the ThC2 joint. The ThC2 joint extends
in stance, producing sideways pushing
motion in stance. All other joints display
similar motion to walking while turning.
This is consistent with what is known
about insects; outside turning motions
usually show little difference from slow
walking in cockroaches (Mu and
Ritzmann 2005) or normal walking in
stick insects (Hellekes et al. 2012).
Figure 35 – CPG output from the middle leg during the
transition to inside turning (top) and outside turning
(bottom). Stance is indicated by gray shading. Turning
is indicated by pink shading.
84
Chapter 5.2.2 – Gait Changes
in the Front Leg
SimROACH can also
produce turning motions with its
front legs. The specific changes
to joint activity that occur to
cause such motion are presented
in (Brown 2011). From these
observations, the stepping rules
for front leg turning in Figure 36
were developed. The networks
in Chapter 4.3.2 – Front Leg
Network encapsulate these rules.
Adapting known rules to
different legs has led to successful walking in robots (Rutter 2010) and biological models
(Ekeberg, Blümel, and Büschges 2004).
Besides changes in joint excursion, outside turning is characterized by changing
the phase of FTi actuation 180 degrees. Inside turning results from changing the phase of
the ThC3 joint by 180 degrees and actuating the ThC2 joint to produce pulling motion.
Figure 38 shows CPG activity during each of these changes. One can see that during
outside turning the FTi joint extends rather than flexing in stance. In addition, the ThC3
joint extends in swing rather than stance to produce inside turning.
Chapter 5.3 – Effect of CPGs on Gait Transitions
FRONT LEG – INSIDE TURNING
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Extend -> Flex
ThC2: Flex -> Extend
ThC3: Flex -> Extend
Fully FLX/Fully DEP CTr: Extend -> Flex
Leg load decreasing ThC3: Extend -> Flex
FTi: Flex -> Extend
Leg unloaded ThC2: Extend -> Flex
Fully EXT CTr: Flex -> Extend
FRONT LEG – OUTSIDE TURNING
Sensory Phenomenon Resulting Phase Changes
Leg loaded FTi: Flex -> Extend
ThC1: Flex -> Extend
ThC3: Extend -> Flex
Fully EXT/Fully DEP CTr: Extend -> Flex
Leg load decreasing ThC3: Flex -> Extend
FTi: Flex -> Extend
Leg unloaded ThC1: Extend -> Flex
Fully FLX CTr: Flex -> Extend
Figure 36 – Tables that show stepping rules for inside turning (top)
and outside turning (bottom) implemented in the front leg of this
model. There is no one authoritative source for these turning rules,
but they are based on literature and hypothesized transitions.
85
The importance of CPGs was demonstrated by comparing gait transitions between
two models, the middle leg of SimROACH and a version without CPGs, similar to
Figure 37 – Control network for the front leg configured to generate inside turning (top) and outside turning
(bottom). The inactive pathways have been only partially filled. The rules for these networks are listed in
Figure 36.
86
LegConNet. The designers of LegConNet
showed that timing was important to
changing gait properly (Rutter et al. 2011).
Experiments with these simulations
confirmed these results.
The single legs were attached to a
simulated cart as described previously.
When made to walk and then transition to
an inside turn, the version with CPGs
successfully transitioned while the version
without ceased stepping 50% of the time
(6 trials). Examining network activity
shows why this occurs. With no CPGs
present, the FTi joint can only flex when
the leg is loaded. In the model with CPGs,
loading reinforces the signal to flex, but
the CPG may cause the transition to occur
slightly before load is detected. While
turning, the leg does not load in the same
manner as during walking, breaking the
reflex cascade. This effect can be seen in
Figure 39. Without CPGs, flexion is only caused by load signals. However, CPGs may
cause the FTi joint to flex before load is detected, making stepping more robust.
Figure 38 – CPG output from the front leg during the
transition to inside turning (top) and outside turning
(bottom). Stance is indicated by gray shading.
Turning is indicated by pink shading.
87
Figure 39 – Plots showing how the command to flex the
FTi joint (green) is only caused by loading (blue) in the
model without CPGs (top), but can precede loading in the
model with CPGs (bottom). Loading then reinforces this
transition, making stepping even more robust.
88
Chapter 6 – Smooth Behavioral Changes
SimROACH also has an intermediate level network that coordinates its legs into
walking gaits. Using Cruse rules (Cruse 1990) and hypothesized interleg pathways
(Daun-Gruhn and Tóth 2010) it can switch between and lock into either a wave gait or a
tripod gait. In addition, the low level networks can be changed by descending commands
to produce turning gaits, much like in stick insects (Hellekes et al. 2012). These features
enable SimROACH to smoothly change between behaviors, something that would benefit
a legged robot.
Chapter 6.1 – Intermediate Level Coordination
The most basic ipsilateral rules are that loading a leg excites unloading of the
anterior leg, and unloading a leg prevents unloading the posterior leg. This rule is the
same as contralateral coupling. These rules coordinate stepping in SimROACH, although
insects use additional rules (See Chapter 2 – Literature Review). SimROACH coordinates
its legs only by coupling the CPG from the CTr joint of each leg to the others.
SimROACH extends the CTr in stance and flexes it during swing in each leg, so coupling
this one CPG keeps enough legs in stance at any time. This minimal coupling distributes
control as much as possible, since each leg manages the details of its own stepping while
sharing only minimal information (CTr state) with the other legs. This scheme is both
flexible and adaptable; the individual legs can change their stepping motions while
maintaining coordination with the others, and each leg can adapt to the terrain
independently of the others.
89
By changing the pathways that couple the legs, SimROACH can produce two
different stepping patterns, a wave gait and a tripod gait. What distinguishes the two? All
Figure 40 – Intermediate level circuit configured to produce a wave gait (A) and a tripod gait (B). Inactive
pathways are shown with less fill. Synapses are color coded according to the key at the bottom.
90
Cruse rules apply in both, so there must be another factor. (Daun-Gruhn and Tóth 2010)
hypothesized that these gaits change in the stick insect due to a modifiable connection
between the hind and front legs. When using the wave gait, signals are passed forward
from leg to leg and then looped around from the front leg to the hind leg to maintain the
even spacing in stepping between legs. To generate a tripod gait, this connection is
changed such that the inhibitory connections become excitatory and vice versa, locking
the stepping phase of the front and hind legs. Cockroaches walk with a wave gait up to a
certain walking speed, at which speed and above they utilize a tripod gait (Bender et al.
2011). SimROACH could explain how the tripod stepping relationship is maintained
even as stepping speed increases.
Gait pathways are switched by the same mechanism by which reflexes are
reversed. Figure 40 (A) shows the circuit in the metachronal wave configuration and
Figure 40B shows the circuit in the tripod configuration. Inhibiting the Metachronal
neuron via descending commands causes the front and hind legs to mirror each other
rather than staggering, producing a tripod gait.
Chapter 6.2 – Intermediate and Low-Level Gait Changes
The wave gait is generated by extending the pattern of excitation and inhibition
between legs to connect the front and hind legs. Figure 41 (A) shows CPG activity from
the wave gait. The half-center of the CPG in each leg (front, middle, hind) associated
with loading the leg (CTr extension) is shown. Each unloads as the leg behind it loads.
This is unremarkable because the tripod gait is also a metachronal gait. What
distinguishes the wave gait is that each leg steps only once before any other leg steps
twice. If lines were drawn connecting the peak activity of each CPG, one could draw a
91
forward slanting or backward
slanting line. But when an insect
walks, it appears to have a forward
traveling wave because each leg
steps exactly once before any leg
steps twice. Therefore drawing a
forward slanting line in Figure 41
(top) makes the most sense for
characterizing a wave gait.
SimROACH can also
produce a tripod gait by coupling the
CPGs in the front and hind legs to
cause simultaneous loading and
unloading. The middle leg steps 180
degrees out of phase of the others
because of the Cruse rules. Figure
41 (bottom) shows CPG output from the model during the tripod gait. Again, only the
half-centers that cause loading are shown. The front and hind legs are clearly in phase
and the middle leg is exactly out of phase. One could draw diagonal lines connecting
peak activity in each CPG, but in such a wave the front and hind legs would step twice
per period, which an observer detects as a distinct pattern.
Stable coordination required careful tuning of synaptic weights between legs. A
numerical simulation performed with XPP revealed that a single CPG oscillates without
Figure 41 – Plots showing CPG activity in the three legs on
one side while walking with a wave gait (top) and a tripod
gait (bottom). The demonstrated patterns are consistent with
gaits seen in insects.
92
any equilibria besides one unstable spiral in the center of the limit cycle. Changing
synaptic conductances between CPGs changes the rate of oscillation of the system. If the
connections are too strong, the eigenvalues of the singular point all become negative and
the point becomes stable, halting oscillation. Therefore SimROACH’s legs are only
weakly connected, but they very rapidly become coordinated, requiring no more than
three or four steps from standstill.
Chapter 6.2.1 – Changing Intermediate
Gait
The front to back connections can
be stably changed without regard to
stepping phase. Even though the transition
momentarily disrupts the ipsilateral
stepping pattern, contralateral leg coupling
ensures SimROACH maintains support of
its thorax. Figure 42 (top) shows CPG
output for ipsilateral and contralateral
CPGs during a gait change. The ipsilateral
coordination smoothly changes by
extending the period of front leg stepping
during the transition. The other legs are
unaffected because the connections
between them do not change. The
disruption of the ipsilateral stepping
pattern is remedied by the contralateral coupling, which ensures one leg of each pair is
Figure 42 – CPG activity during the transition from a
wave gait to tripod gait in ipsilateral (top) and
contralateral (bottom) legs. The first trace is the same
in each plot. Tripod walking and the transition are
highlighted in pink.
93
always on the ground. Figure 41 (bottom) shows CPG output from both front legs during
the same transition. Whenever the left leg extends its CTr joint, the right leg flexes its
CTr joint. This coupling scheme ensures that SimROACH does not fall over while
transitioning from a wave to a tripod gait.
Chapter 6.2.2 – Changing Low Level Gait
Chapter 5.2 – Flexible Networks Capable of Changing Gait described how single
legs of SimROACH can switch between walking and turning by reversing reflexes and
deactivating joints. How do these changes affect the behavior of the entire system?
SimROACH is a massively distributed control system, so while individual joints change
during gait transitions, the rest of the system should be unaffected. Results show that this
is true.
SimROACH turns by stimulating the
neurons that code for inside turn in the front
and middle legs of one side and stimulating the
neurons that code for outside turn in the front
and middle legs of the other side. This is
accomplished by stimulating all of the intended
turning neurons by one neuron that codes for
turning right or left, as shown in Figure 43. For
example, Turn Right will excite the Inside Turn neurons in the right legs and the Outside
Turn neurons in the left legs. The hind legs to not change their gait during turns, as in
Blaberus (Mu and Ritzmann 2005).
As intended, the effect of such changes on the intermediate level control system is slight.
Figure 44 shows CPG output during turning while using the tripod and wave gaits. Since
Figure 43 – Picture of a segment of the
intermediate circuit configured to turn right by
stimulating the Turn Right neuron, which in
turn stimulates the proper low level turning
neurons.
94
the intermediate level gait is not
affected by turning behavior,
coordination is maintained in both
configurations. As noted previously,
the legs are only coupled through the
CPG that controls the CTr joints
because they extend in stance and flex
in swing in every leg during every
gait. Therefore the phase relationship
between the CTr in each leg should
not change while turning.
Maintaining coordination
allows SimROACH to produce
turning behavior. Experiments were
performed to quantify direction
changes when the command to turn
was given. Its path was recorded and
the curvature was calculated as a function of path traveled. Results were gathered for
both the wave and tripod gaits. SimROACH walked forward for 5 seconds and then
turned for 5 seconds. Figure 45 shows results from two trials, one right and one left turn,
showing clear changes in behavior as a result of the low level stepping rule changes. The
RMS path curvature for all data using the tripod and wave gaits was 23.77 m-1
and 21.54
Figure 44 – Plots of CPG activity during the transition from
forward walking to turning while using the wave gait (top)
and the tripod gait (bottom). Turning is highlighted in pink.
Dotted lines show that coordination is maintained during
the transition.
95
m-1
, respectively. This suggests that there was little difference between the performances
when no sensory information was incorporated into intermediate level coordination.
The radius of curvature varies greatly while turning, an undesirable trait for an
engineered system. Videos of turning experiments reveal missteps in which a leg does not
load properly, pulling at the air or brushing the ground. Another version of SimROACH’s
intermediate level circuit, shown in Figure 46, was developed in which pathways were
gated by loading information. This gating did not make a noticeable difference in turning
performance. Other models connect legs by allowing sensory information to modify
activity of the low level circuits in adjacent legs (Daun-Gruhn 2010). Similar work with
mammalian system modeling in the Biologically Inspired Robotics Lab has produced
effective interleg coupling based on the same principle (Alex Hunt, Unpublished
Figure 45 – Robot heading (top) during two typical turning trials. The robot is commanded to walk straight
for 5 s (blue) and then turn (green). The paths were smoothed with a Gaussian kernel, and the curvature
(bottom) for each trial was calculated as a function of path length. In the left turn trial, the RMS curvature
was 5.484 during forward walking and 28.83 during turning. In the right turn trial, the RMS curvature was
6.317 during walking and 25.01 during turning.
96
Results). Implementing similar rules in the future might improve SimROACH’s
performance.
Being able to change the radius of curvature would also be important for an actual
robot. Currently SimROACH simply produces turning motions with each leg in an
untargeted way. Perhaps ThC2 actuation, which controls abduction and adduction of each
leg, could be modulated to produce turning motions that are more or less severe.
Figure 46 – Intermediate level circuit modified to require loading information to tell the ipsilateral leg to
unload. This sensory information is only utilized during the metachronal wave gait.
97
Chapter 7 – Conclusions and Future Work
Chapter 7.1 – Conclusions
This thesis presents a massively distributed control system based in insect neurobiology,
SimROACH, which controls stepping in both software and hardware robot legs. The
entire control network is assembled from physiological neuron and synapse models,
meaning that sensory pathways and CPGs can be implemented in a biologically plausible
way. This does not make SimROACH exactly like an animal, but since animals are much
better locomotors than robots it is hoped this more accurate biomimicry will improve a
robot’s walking ability. SimROACH is able to change gaits smoothly and stably,
something that remains a challenge for some robots today. Perhaps more biological
accuracy in the future will further improve its performance.
SimROACH also represents an alternative to traditional centralized robotic control
methods. Like other distributed and neural network control systems, this network may
solve the same problems in a more efficient way. Rather than performing complicated
mathematical operations to set actuator torques, SimROACH only uses a computer to
compute simple integration schemes, which are much less computationally expensive.
The network connectivity determines the behavior and while more complicated behavior
will require a larger network, the simulation method will not become more complicated.
With further development and the addition of more advanced computation hardware like
FPGAs, SimROACH and related systems may become an attractive method for
controlling walking robots in the future.
SimROACH was largely successful in accomplishing its goals. The first primary goal
was to produce robust walking motions. SimROACH used a simulated nervous system to
generate walking with structures discovered or hypothesized in stick insects and
98
cockroaches. The resulting motion is robust to perturbation and certainly carries
SimROACH forward, although the kinematics do not precisely match all aspects of
Blaberus, the primary model organism. Several steps in the future work outline how this
could be improved.
In addition walking, SimROACH can smoothly transition between walking and turning
behaviors. It models hypothesized connections in insect nervous systems that allow them
to make small changes to interjoint coordination and produce different stepping motions.
This approach has been successful in simulation and will soon be applied to hardware.
SimROACH was also moderately successful in becoming a useful model of insect
locomotion control. SimROACH coordinates its legs in ways known or hypothesized to
exist in insects, and captures a lot of what the animal does. The apparent motion is not
identical, but this may be due to biological testing conditions or the simulation
environment. In addition, such differences may lead to testable hypotheses for future
biological research, such as finding pathways that appear to be necessary for proper
motion in the model. In spite of this, the parameters of SimROACH’s nervous system
could be improved, and the future work is focused on resolving these issues. Better
tuning could yield both more successful walking and more accurate biological models.
Chapter 7.2 – Future Work
Chapter 7.2.1 – Sensitivity Analysis and Parameter Tuning
SimROACH’s locomotion, particularly the motion of its joints, does not precisely
match that of the model organism Blaberus discoidalis. SimROACH simplifies many
aspects of neurobiology, but proper joint range of motion is a straightforward comparison
metric and should be obtainable despite simplification. Numerical tuning of muscle and
neuron properties is largely responsible for these discrepancies. A formal sensitivity
99
analysis has not been performed on this system, but this will be crucial to direct any
attempt to optimize SimROACH. Other work in the Biologically Inspired Robotics
Laboratory has performed sensitivity analysis on muscle models, which could be
leveraged in SimROACH.
Earlier in this project a Matlab program was written that explored neural behavior by
constructing a network, simulating its behavior for a short time, varying parameters of
neurons and synapses, analyzing the output from each case, and fitting the results to a
hypersurface for optimization. This approach ultimately failed because it explored
parameters by generating every permutation of the system given ranges and resolutions
for parameters, which both used too much memory and was time consuming. However,
this experimentation made it clear that component and system behavior were more
sensitive to some parameters than others, motivating a more formal sensitivity analysis of
neural models in the future.
What kind of tuning technique would be more appropriate for a system of this type and
scale? Several methods exist for solving this type of problem, and fortunately this system
has a reasonable seed value (SimROACH in its current form) and “optimal” data
(kinematic data from the Ritzmann lab). Back propagation could be developed for these
neuron and synapse models, although such a method may be slow for a system of this
size. It may be useful to divide the system into subsystems and train each piece
separately. Many sophisticated genetic algorithms exist, but finding a suitable parameter
set for a system of this size may be very time consuming. More research and
experimentation will have to be done to find a suitable method for tuning.
100
Chapter 7.2.2 – Actuator Types
System performance may also be improved by using a different actuator than the
simulated muscle used in SimROACH. SimROACH’s two primary actuator issues are
improper walking kinematics and no flexibility in walking speed. The range of motion
depends on the length-tension relationship of the muscles, which has been very difficult
to tune. An automated tuning method, if developed, could resolve this issue. However,
the robot’s servos produced the desired ranges of motion by adapting the muscle control
units as described in Chapter 4.5 – Robotic Implementation. This took very little time to
implement, suggesting that eliminating muscles would accelerate the development of any
future system based on SimROACH.
Despite this success, the resulting motion was somewhat linear and inorganic.
Rather than servos, a properly tuned muscle model with slow and fast muscle fibers could
potentially produce motion closer to that seen in animals. Assuming tuning could produce
the proper range of motion, the inclusion of additional fibers would enable SimROACH
to generate more torque at its joints during transitions between stance and swing,
increasing its walking speed like a cockroach (Watson and Ritzmann 1998). In addition,
continuing to use simulated muscles would allow more direct comparisons between
SimROACH and insects than servos would.
Chapter 7.2.3 – Intermediate Circuit
Besides muscles and parameter tuning, the completeness of the intermediate
network could be improved. SimROACH uses a simple set of Cruse rules without any
sensory feedback. SimROACH’s walking has not been tested over rugged terrain, but one
would expect it to struggle due to the lack of sensory feedback in the interleg
connections. Simply adding more connections from sensors to CPGs caused the CPGs to
101
stop oscillating, similar to the phenomenon discussed in Chapter 6.2 – Intermediate and
Low-Level Gait Changes. Successfully implementing such changes would require a
holistic design, that is, adding all connections simultaneously with low synaptic strength.
This is very difficult to do properly by hand.
Another alternative would be to couple the legs by allowing sensory information
from one leg to modulate the muscle control units or sensory information in another. This
could be used to make muscle positional or stiffness modifications, which are also
important to coordinating multiple legs, rather than only step timing changes. This would
be particularly useful when turning because different legs of SimROACH seem to fight
each other when the simulation turns. Such additions should not halt CPG oscillation
since the CPGs would not be directly affected. Features like these have led to successful
interleg coupling in stick insect modeling (Daun-Gruhn 2010).
Chapter 7.2.4 – Robotic Leg
The final major improvement relates to the robotic leg. Currently the neural
dynamics are calculated on a laptop nearby the test stand. Adding more neurons, whether
for additional features or other legs, will increase the number of calculations performed
every time step, slowing the system down. This method could not be used to control a
mobile robot due to the power consumption and weight of the computer. Currently, field
programmable gate arrays (FPGA) are being examined as an alternative. FPGAs are
control chips that can be physically rewired by a computer. This builds the desired
functionality into a circuit, allowing the neural system to be constructed in hardware and
run with each neuron in parallel. Preliminary tests with the computing hardware in
Chapter 4.5 – Robotic Implementation suggest that the FPGA is over 25 times faster than
a typical duo-core processor. In addition, building a larger network does not slow the
102
FPGA down, since it physically constructs a separate circuit for each neuron, then
simulates them in parallel. This technology shows great promise for the simulation of
physiological neural systems onboard robots.
103
Appendix A – Network Topologies
Since this work is ultimately for the development of an engineered device, the
permutations of properties for neurons and synapses was kept to a minimum. Therefore a
few stereotypical property combinations have been provided. Otherwise, the properties of
each are listed next to their location on the maps. For nonspiking neurons, properties are
listed in order resting voltage, time constant, membrane noise, tonic current, maximum
calcium conductance, calcium activation midpoint voltage, calcium activation slope,
calcium activation time constant, calcium deactivation midpoint voltage, calcium
deactivation slope, and calcium deactivation time constant. Spiking neurons’ properties
are listed as resting voltage, spiking threshold, membrane noise, tonic current, spiking
threshold accommodation, and threshold accommodation time constant. Nonspiking
synapses are listed as equilibrium potential, maximum conductance, low conductance
threshold, and high conductance threshold. Finally, spiking synapses are listed as
equilibrium potential, maximum conductance, time constant, facilitation, and facilitation
time constant.
Standard NSN -60 mV, 5 ms, 0 mV, 0 nA
Standard CaNSN -60 mV, 5 ms, 0.1 mV, 0 nA, 5 uS, -40 mV, 0.1, 2 ms, -100
mV, -0.1, 250 ms
Standard Depolarizing -40 mV, 2 uS, -60 mV, -40 mV
Threshold Depolarizing -40 mV, 2 uS, -47 mV, -45 mV
Post Gate Depolarizing -40 mV, 2 uS, -60 mV, -50 mV
104
Front Leg
Location Properties
F1 -60 mV, -58 mV, 0 mV, 0 nA, 0.5, 10 ms
D2 -60 mV, -55 mV
E2 -60 mV, -58 mV, 0 mV, 6 nA
2 (G to I), 3, 4, 5 (All) Standard NSN
6 (All) Standard CaNSN
7 (All) Standard NSN
8 (All) -50 mV
9 (All) -100 mV, 20 ms
10 (All) -50 mV
F1 to E2 -70 mV, 1 uS, 10 ms, 1, 50 ms
E2 to D2 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
D2 to F2 Standard Depolarizing
D2 to C3 Standard Depolarizing
E2 to D2 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
E2 to A4 Standard Depolarizing
F2 to I4 Standard Depolarizing
F2 to J4 Standard Depolarizing
G2 to D3 Standard Depolarizing
G2 to E3 Standard Depolarizing
G2 to F4 Standard Depolarizing
G2 to H3 Standard Depolarizing
G2 to I3 Standard Depolarizing
H2 to G4 Standard Depolarizing
105
H2 to H4 Standard Depolarizing
I2 to F3 Standard Depolarizing
I2 to G3 Standard Depolarizing
C3 to C4 Standard Depolarizing
C3 to D4 Standard Depolarizing
C3 to E5 Standard Depolarizing
D3 to D5 Post Gate Depolarizing
E3 to C5 Threshold Depolarizing
F3 to H5 Post Gate Depolarizing
G3 to G5 Threshold Depolarizing
H3 to J5 Post Gate Depolarizing
A4 to A5 Threshold Depolarizing
B4 to B5 Threshold Depolarizing
C4 to C5 Post Gate Depolarizing
D4 to D5 Threshold Depolarizing
E4 to E5 Post Gate Depolarizing
F4 to F5 Post Gate Depolarizing
G4 to G5 Post Gate Depolarizing
H4 to H5 Threshold Depolarizing
I4 to I5 Post Gate Depolarizing
J4 to J5 Threshold Depolarizing
5 to 6 (All) -70 mV, 2 uS, -60 mV, -40 mV
6 to 5 (All) Standard Depolarizing
8 to 7 (All) -80 mV, 2 uS, -60 mV, -20 mV
7 to 9 (All) -10 mV, 1 uS, -60 mV, -20 mV
8 to 8 (All) Standard Depolarizing
10 to 8 (All) -80 mV, 2 uS, -60 mV, -20 mV
Standing to 6 (All) -70 mV, 2 uS, -60 mV, -40 mV
Outside turning to F3, G4, H3, I4 -90 mV, 3 uS, -60 mV, -40 mV
Outside turning to G3, H4, I3 Standard Depolarizing
Inside turning to C4, D3, E4, F4 -90 mV, 3 uS, -60 mV, -40 mV
Inside turning to A4, B4, D4, E3 Standard Depolarizing
Walking 2 to A6, B6 -80 mV, 1 uS, 20 mS, 1, 50 ms
106
Middle Leg
Location Properties
D1 -60 mV, -58 mV, 0 mV, 6 nA
F1 -60 mV, -58 mV, 0 mV, 0 nA, 0.5, 10 ms
D2 -60 mV, -55 mV, 0 mV, 0 nA
2 (A to C, E to J), 3, 4 (All) Standard NSN
5 (All) Standard CaNSN
6 (All) Standard NSN
7 (All) -50 mV
8 (All) -100 mV, 20 ms
9 (All) -50 mV
D1 to A2 Standard Depolarizing
D1 to B3 Standard Depolarizing
D1 to D2 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
D1 to J2 Standard Depolarizing
F1 to D1 -70 mV, 1 uS, 10 ms, 1, 50 ms
F1 to F2 -10 mV, 1 uS, 20 ms, 0.5, 50 ms
F1 to G2 -10 mV, 0.5 uS, 3 ms, 1, 100 ms
A2 to A4 Threshold Depolarizing
B2 to B4 Threshold Depolarizing
C2 to B3 Standard Depolarizing
C2 to H4 Post Gate Depolarizing
D2 to E2 Standard Depolarizing
E2 to F4 Standard Depolarizing
F2 to G4 Standard Depolarizing
G2 to G3 Standard Depolarizing
G2 to H3 Standard Depolarizing
107
H2 to C3 Standard Depolarizing
H2 to D3 Standard Depolarizing
I2 to E3 Standard Depolarizing
I2 to F3 Standard Depolarizing
J2 to I3 Standard Depolarizing
J2 to J3 Standard Depolarizing
A3 to A4 Threshold Depolarizing
B3 to B4 Threshold Depolarizing
C3 to H4 Post Gate Depolarizing
D3 to E4 Threshold Depolarizing
E3 to E4 Post Gate Depolarizing
F3 to H4 Threshold Depolarizing
G3 to I4 Post Gate Depolarizing
H3 to J4 Threshold Depolarizing
I3 to J4 Post Gate Depolarizing
J3 to I4 Threshold Depolarizing
4 to 5 (All) -70 mV, 2 uS, -60 mV, -40 mV
5 to 4 (All) Standard Depolarizing
5 to 6 (All) -70 mV, 2 uS, -60 mV, -40 mV
6 to 8 (All) -10 mV, 1 uS, -60 mV, -20 mV
7 to 6 (All) -10 mV, 1 uS, -60 mV, -20 mV
7 to 7 (All) Standard Depolarizing
9 to 7 (All) -80 mV, 2 uS, -60 mV, -40 mV
Inside Turning to A2, B2, D3, F3,
H3, J3
Standard Depolarizing
Inside Turning to C3, E3, G3, I3 -90 mV, 3 uS, -60 mV, -40 mV
Inside Turning to Walking 2 -90 mV, 3 uS, -60 mV, -40 mV
Outside Turning to A3, B3 Standard Depolarizing
Outside Turning to Walking 2 -90 mV, 3 uS, -60 mV, -40 mV
Walking 2 to A5, B5 -80 mV, 1 uS, 20 ms, 1, 50 ms
Walking 1 to C5, D5 -80 mV, 1 uS, 20 ms, 1, 50 ms
Standing to 5 (All) -70 mV, 2 uS, -60 mV, -40 mV
108
Hind Leg
H1 -60 mV, -58 mV, 0 mV, 0 nA, 0.5, 10 ms
A2 -60 mV, -55 mV, 0 mV, 7 nA
C2 -60 mV, -55 mV, 0 mV, 7 nA
F2 -60 mV, -55 mV
G2 -60 mV, -58 mV, 0 mV, 6 nA
2 (H, I), 3, 5 (All) Standard NSN
4 (All) Standard CaNSN
6 (All) -50 mV
7 (All) -100 mV, 20 ms
8 (All) -50 mV
H1 to G2 -70 mV, 1 uS, 10 ms, 1, 50 ms
G2 to F2 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
G2 to H3 Standard Depolarizing
G2 to J3 Standard Depolarizing
F2 to H2 Standard Depolarizing
A2 to A4 -70 mV, 1 uS, 20 ms, 1, 50 ms
A2 to B4 -70 mV, 1 uS, 20 ms, 1, 50 ms
C2 to C4 -70 mV, 1 uS, 20 ms, 1, 50 ms
C2 to D4 -70 mV, 1 uS, 20 ms, 1, 50 ms
H2 to F3 Standard Depolarizing
H2 to H3 Standard Depolarizing
I2 to G3 Standard Depolarizing
I2 to I3 Standard Depolarizing
3 to 4 (All) -70 mV, 2 uS, -60 mV, -40 mV
4 to 3 (All) Standard Depolarizing
4 to 5 (All) -70 mV, 2 uS, -60 mV, -40 mV
109
6 to 5 (All) -10 mV, 2 uS, -60 mV, -20 mV
5 to 7 (All) -10 mV, 1 uS, -60 mV, -20 mV
6 to 6 (All) Standard Depolarizing
8 to 6 (All) -80 mV, 2 uS, -60 mV, -40 mV
Standing to 4 (All) -70 mV, 2 uS, -60 mV, -40 mV
110
Intermediate Level Circuit
111
1 -60 mV, 500 ms, 0 mV, 10 nA
B2 -60 mV, -55 mV
C2 Standard NSN
D2 Standard CaNSN
E2 Standard CaNSN
F2 Standard NSN
G2 -60 mV, -55 mV
A3 Standard NSN
B3 -60 mV, -55 mV
G3 Standard NSN
H3 Standard NSN
A4 Standard NSN
B4 Standard NSN
G4 -60 mV, -55 mV
H4 Standard NSN
A5 Standard NSN
B5 Standard NSN
G5 Standard NSN
H5 Standard NSN
A6 Standard NSN
B6 -60 mV, -55 mV
G6 -60 mV, -55 mV
H6 Standard NSN
B7 Standard NSN
G7 Standard NSN
8 -60 mV, -55 mV
12 Standard NSN
14 Standard NSN
1 to A3 -90 mV, 3 uS, -60 mV, -40 mV
1 to A4 Standard Depolarizing
1 to A5 -90 mV, 3 uS, -60 mV, -40 mV
1 to A6 Standard Depolarizing
1 to H3 -90 mV, 3 uS, -60 mV, -40 mV
1 to H4 Standard Depolarizing
1 to H5 -90 mV, 3 uS, -60 mV, -40 mV
1 to H6 Standard Depolarizing
B2 to B3 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
D2 to F2 -70 mV, 0.05 uS, -60 mV, -40 mV
E2 to C2 -70 mV, 0.05 uS, -60 mV, -40 mV
G2 to G3 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
A3 to C2 -40 mV, 0.1 uS, -48 mV, -46 mV
B3 to C4 -40 mV, 0.1 uS, -48 mV, -46 mV
D3 to B2 Standard Depolarizing
D3 to F2 -40 mV, 0.05 uS, -60 mV, -40 mV
E3 to C2 -40 mV, 0.05 uS, -60 mV, -40 mV
112
E3 to G2 Standard Depolarizing
G3 to F4 -40 mV, 0.1 uS, -48 mV, -46 mV
H3 to F2 -40 mV, 0.1 uS, -48 mV, -46 mV
A4 to C2 -70 mV, 0.1 uS, -48 mV, -46 mV
B4 to B5 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
D4 to F4 -70 mV, 0.05 uS, -60 mV, -40 mV
E4 to C4 -70 mV, 0.05 uS, -60 mV, -40 mV
G4 to G5 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
H4 to F2 -70 mV, 0.1 uS, -48 mV, -46 mV
A5 to C4 -40 mV, 0.1 uS, -48 mV, -46 mV
B5 to C6 -40 mV, 0.1 uS, -48 mV, -46 mV
D5 to B4 Standard Depolarizing
D5 to F4 -40 mV, 0.05 uS, -60 mV, -40 mV
E5 to C4 -40 mV, 0.05 uS, -60 mV, -40 mV
E5 to G4 Standard Depolarizing
G5 to F6 -40 mV, 0.1 uS, -48 mV, -46 mV
H5 to F4 -40 mV, 0.1 uS, -48 mV, -46 mV
A6 to C2 -50 mV, 0.1 uS, -48 mV, -46 mV
B6 to B7 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
D6 to A3 Standard Depolarizing
D6 to A4 Standard Depolarizing
E6 to H3 Standard Depolarizing
E6 to H4 Standard Depolarizing
G6 to G7 -10 mV, 0.5 uS, 20 ms, 0.5, 50 ms
B7 to A5 Standard Depolarizing
B7 to A6 Standard Depolarizing
D7 to B6 Standard Depolarizing
D7 to F6 -40 mV, 0.05 uS, -60 mV, -40 mV
E7 to C6 -40 mV, 0.05 uS, -60 mV, -40 mV
E7 to G6 Standard Depolarizing
G7 to H5 Standard Depolarizing
G7 to H6 Standard Depolarizing
8 to C9, F9, C10, F10, C11, F11 -100 mV, 3 uS, 20 ms, 1, 50 ms
12 to C13, D13, E13, F13 Standard Depolarizing
14 to C15, D15, E15, F15 Standard Depolarizing
113
Bibliography
Akay, Turgay, U Bässler, Petra Gerharz, and Ansgar Büschges. 2001. “The Role of Sensory
Signals from the Insect Coxa-trochanteral Joint in Controlling Motor Activity of the Femur-
tibia Joint.” Journal of Neurophysiology 85 (2): 594.
Akay, Turgay, and Ansgar Büschges. 2006. “Load Signals Assist the Generation of Movement-
dependent Reflex Reversal in the Femur-tibia Joint of Stick Insects.” Journal of
Neurophysiology 96 (6) (December): 3532–7.
Akay, Turgay, Sebastian Haehn, Josef Schmitz, and Ansgar Büschges. 2004. “Signals from Load
Sensors Underlie Interjoint Coordination During Stepping Movements of the Stick Insect
Leg.” Journal of Neurophysiology 92 (1) (July): 42–51.
Akay, Turgay, Björn C Ludwar, Marie L Göritz, Josef Schmitz, and Ansgar Büschges. 2007.
“Segment Specificity of Load Signal Processing Depends on Walking Direction in the Stick
Insect Leg Muscle Control System.” The Journal of Neuroscience 27 (12) (March): 3285–
94.
Arbas, Edmund A, and Ronald L Calabrese. 1987. “Slow Oscillations of Membrane Potential in
Interneurons That Control Heartbeat in the Medicinal Leech.” Journal of Neuroscience 7
(December 1987): 3953–3960.
Beer, Randall D, Hillel J Chiel, Roger D Quinn, and Kenneth S Espenschied. 1992. “A
Distributed Neural Network Architecture for Hexapod Robot Locomotion.” Neural
Computation 4: 356–365.
Bender, John A, Elaine M Simpson, and Roy E Ritzmann. 2010. “Computer-assisted 3D
Kinematic Analysis of All Leg Joints in Walking Insects.” PloS One 5 (10) (January).
Bender, John A, Elaine M Simpson, Brian R Tietz, Kathryn A Daltorio, Roger D Quinn, and Roy
E Ritzmann. 2011. “Kinematic and Behavioral Evidence for a Distinction Between Trotting
and Ambling Gaits in the Cockroach Blaberus Discoidalis.” Journal of Experimental
Biology 214 (12) (May 25): 2057–2064.
Brown, Amy E. 2011. “Multi-Legged Joint Kinematic Analysis of an Insect Tethered Over a
Slippery Surface”.
Bucher, Dirk, Turgay Akay, Ralph A DiCaprio, and Ansgar Büschges. 2003. “Interjoint
Coordination in the Stick Insect Leg-control System: The Role of Positional Signaling.”
Journal of Neurophysiology 89 (3): 1245–1255.
Büschges, Ansgar. 1995. “Role of Local Nonspiking Interneurons in the Generation of Rhythmic
Motor Activity in the Stick Insect.” Journal of Neurobiology 27 (4): 488–512.
Büschges, Ansgar, R Kittmann, and Josef Schmitz. 1994. “Identified Nonspiking Interneurons in
Leg Reflexes and During Walking in the Stick Insect.” Journal of Comparative Physiology
A: …: 685–700.
114
Büschges, Ansgar, Josef Schmitz, and U Bässler. 1995. “Rhythmic Patterns in the Thoracic Nerve
Cord of the Stick Insect Induced by Pilocarpine.” Journal of Experimental Biology 198 (Pt
2): 435–456.
Büschges, Ansgar, Turgay Akay, Jens P Gabriel, and Joachim Schmidt. 2008. “Organizing
Network Action for Locomotion: Insights from Studying Insect Walking.” Brain Research
Reviews 57 (1): 162–171.
Büschges, Ansgar, Björn C Ludwar, Dirk Bucher, Joachim Schmidt, and Ralph a DiCaprio. 2004.
“Synaptic Drive Contributing to Rhythmic Activation of Motoneurons in the Deafferented
Stick Insect Walking System.” The European Journal of Neuroscience 19 (7) (April): 1856–
62.
Carbonell, C.S. 1947. The Thoracic Muscles of the Cockroach Periplaneta Americana (L.). Vol.
107. Washington, D.C.: Smithsonian Institution.
Cheung, O.Y.H., P.H.W. Leong, E.K.C. Tsang, and B.E. Shi. 2006. “A Scalable FPGA
Implementation of Cellular Neural Networks for Gabor-type Filtering.” The 2006 IEEE
International Joint Conference on Neural Network Proceedings: 15–20.
Chou, Shih-Wei, Hsin-Yi Kathy Cheng, Jen-Hua Chen, Yan-Ying Ju, Yin-Chou Lin, and May-
Kuen Alice Wong. 2009. “The Role of the Great Toe in Balance Performance.” Journal of
Orthopaedic Research : Official Publication of the Orthopaedic Research Society 27 (4)
(April): 549–54.
Cofer, David W. 2009. “Neuromechanical Analysis of Locust Jumping.”
Cofer, David W, Gennady Cymbalyuk, James Reid, Ying Zhu, William J Heitler, and Donald H
Edwards. 2010. “AnimatLab: a 3D Graphics Environment for Neuromechanical
Simulations.” Journal of Neuroscience Methods 187 (2) (March 30): 280–8.
Cruse, Holk. 1990. “What Mechanisms Coordinate Leg Movement in Walking Arthropods?”
Trends in Neurosciences 13 (1990): 15–21.
Cruse, Holk, Thomas Kindermann, Michael Schumm, Jeffrey Dean, and Josef Schmitz. 1998.
“Walknet-a Biologically Inspired Network to Control Six-legged Walking.” Neural
Networks : the Official Journal of the International Neural Network Society 11 (7-8)
(October): 1435–1447.
Daun-Gruhn, Silvia. 2010. “A Mathematical Modeling Study of Inter-segmental Coordination
During Stick Insect Walking.” Journal of Computational Neuroscience (June): 255–278.
Daun-Gruhn, Silvia, and Tibor Istvan Tóth. 2010. “An Inter-segmental Network Model and Its
Use in Elucidating Gait-switches in the Stick Insect.” Journal of Computational
Neuroscience (December).
Ekeberg, Orjan, Marcus Blümel, and Ansgar Büschges. 2004. “Dynamic Simulation of Insect
Walking.” Arthropod Structure & Development 33 (3) (July): 287–300.
115
Espenschied, Kenneth S, Roger D Quinn, Randall D Beer, and Hillel J Chiel. 1996. “Biologically
Based Distributed Control and Local Reflexes Improve Rough Terrain Locomotion in a
Hexapod Robot.” Robotics and Autonomous Systems 18 (1-2) (July): 59–64.
Espenschied, Kenneth S, Roger D Quinn, Hillel J Chiel, and Randall D Beer. 1995.
“Biologically-Inspired Hexapod Robot Project: Robot II.” In IEEE International
Conference on Robotics and Automation Video Proceedings. Nagoya, Japan.
Full, Robert J, Reinhard Blickhan, and L H Ting. 1991. “Leg Design in Hexapedal Runners.” The
Journal of Experimental Biology 158 (July): 369–90.
Gruhn, Matthias, Lyuba Zehl, and Ansgar Büschges. 2009. “Straight Walking and Turning on a
Slippery Surface.” Journal of Experimental Biology 212 (Pt 2): 194–209.
Hellekes, K., E. Blincow, J. Hoffmann, and A. Büschges. 2011. “Control of Reflex Reversal in
Stick Insect Walking: Effects of Intersegmental Signals, Changes in Direction and
Optomotor Induced Turning.” Journal of Neurophysiology (October 12).
Hellekes, Katja, Eric Blincow, Julia Hoffmann, and Ansgar Büschges. 2012. “Control of Reflex
Reversal in Stick Insect Walking: Effects of Intersegmental Signals, Changes in Direction,
and Optomotor-induced Turning.” Journal of Neurophysiology 107 (1) (January): 239–49.
Hess, Dietmar, and Ansgar Büschges. 1999. “Role of Proprioceptive Signals from an Insect
Femur-tibia Joint in Patterning Motoneuronal Activity of an Adjacent Leg Joint.” Journal of
Neurophysiology 81 (4): 1856–1865.
Hess, Dietmar, and Ansgar Büschges. 1999. “Role of Proprioceptive Signals From an Insect
Femur-Tibia Joint in Patterning Motoneuronal Activity of an Adjacent Leg Joint.” Journal
of Neurophysiology 81: 1856–1865.
Hodgkin, A L, A F Huxley, and B Katz. 1952. “Measurement of Current-Voltage Relations in the
Membrane of the Giant Axon of Loligo.” J. Physiol. 116: 442–448.
Izhikevich, Eugene M. 2004. “Which Model to Use for Cortical Spiking Neurons?” IEEE
Transactions on Neural Networks / a Publication of the IEEE Neural Networks Council 15
(5) (September): 1063–70.
Jindrich, Devin L, and Robert J Full. 2002. “Dynamic Stabilization of Rapid Hexapedal
Locomotion.” The Journal of Experimental Biology 205 (Pt 18) (September): 2803–23.
Kimura, H, Y Fukuoka, and A H Cohen. 2007. “Adaptive Dynamic Walking of a Quadruped
Robot on Natural Ground Based on Biological Concepts.” The International Journal of
Robotics Research 26 (5): 475–490.
Kingsley, Daniel, Roger D Quinn, and Roy E Ritzmann. 2006. “A Cockroach Inspired Robot
With Artificial Muscles.” 2006 IEEE/RSJ International Conference on Intelligent Robots
and Systems (October): 1837–1842.
116
Lewinger, William A, and Brandon L Rutter. 2006. “Sensory Coupled Action Switching Modules
(SCASM) Generate Robust, Adaptive Stepping in Legged Robots.” Climbing and Walking
Robots (September).
Lewinger, William A, and Roger D Quinn. 2010. “A Hexapod Walks Over Irregular Terrain
Using a Controller Adapted from an Insect’s Nervous System.” IEEE Int Conf on Intelligent
Robots and Systems: 3386–3391.
Loeb, E, E Brown, and J Cheng. 1999. “A Hierarchical Foundation for Models of Sensorimotor
Control.” Experimental Brain Research: 1–18.
Mackay-lyons, Marilyn. 2002. “Spinal Cord Injury Special Series Central Pattern Generation of
Locomotion : A Review of The.” Americal Physical Therapy Association 82: 69–83.
Manoonpong, Poramate, Frank Pasemann, and Worgotter Florentin. 2007. “Reactive Neural
Control for Phototaxis and Obstacle Avoidance Behavior of Walking Machines.”
International Journal of … 1 (3): 172–177.
Meyer, Peter F, Lars I E Oddsson, and Carlo J De Luca. 2004. “Reduced Plantar Sensitivity
Alters Postural Responses to Lateral Perturbations of Balance.” Experimental Brain
Research. Experimentelle Hirnforschung. Expérimentation Cérébrale 157 (4) (August):
526–36.
Mu, Laiyong, and Roy E Ritzmann. 2005. “Kinematics and Motor Activity During Tethered
Walking and Turning in the Cockroach, Blaberus Discoidalis.” Journal of Comparative
Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 191 (11)
(November): 1037–54.
Mu, Laiyong, and Roy E Ritzmann. 2008. “Interaction Between Descending Input and Thoracic
Reflexes for Joint Coordination in Cockroach: I. Descending Influence on Thoracic Sensory
Reflexes.” Journal Of Comparative Physiology A Neuroethology Sensory Neural And
Behavioral Physiology 194 (3): 283–98.
Nelson, Gabriel M, Roger D Quinn, Richard J Bachmann, W C Flannigan, Roy E Ritzmann, and
James T Watson. 1997. “Design and Simulation of a Cockroach-like Hexapod Robot.” In
Proceedings of 1997 IEEE International Conference on Robotics and Automation, 3–8.
Noah, J A, L Quimby, S F Frazier, and Sasha N Zill. 2004. “Walking on a ‘Peg Leg’: Extensor
Muscle Activities and Sensory Feedback After Distal Leg Denervation in Cockroaches.”
Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral
Physiology 190 (3) (March): 217–31.
Pearson, K G. 1993. “Common Principles of Motor Control in Vertebrates and Invertebrates.”
Annual Review of Neuroscience 16 (1): 265–297.
Rassier, D E, B R MacIntosh, and W Herzog. 1999. “Length Dependence of Active Force
Production in Skeletal Muscle.” Journal of Applied Physiology (Bethesda, Md. : 1985) 86
(5) (May): 1445–57.
117
Ridgel, Angela L, S Faith Frazier, Ralph A Dicaprio, Sasha N Zill, Bridget R Keller, and
Elizabeth R Duke. 1999. “Active Signaling of Leg Loading and Unloading in the Cockroach
Active Signaling of Leg Loading and Unloading in the Cockroach.” Journal of
Neurophysiology 81: 1432–1437.
Ridgel, Angela L, and Roy E Ritzmann. 2005. “Effects of Neck and Circumoesophageal
Connective Lesions on Posture and Locomotion in the Cockroach.” Journal of Comparative
Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology 191 (6) (June):
559–73.
Rutter, Brandon L, Brian K Taylor, John A Bender, M Blumel, William A Lewinger, Roy E
Ritzmann, and Roger D Quinn. 2011. “Descending Commands to an Insect Leg Controller
Network Cause Smooth Behavioral Transitions.” Intelligent Robots and Systems (IROS
2011).
Rutter, Brandon L. 2010. “Robotic Models of Neuromechanical Step Generation in Insects”.
Ryckebusch, S, and G Laurent. 1993. “Rhythmic Patterns Evoked in Locust Leg Motor Neurons
by the Muscarinic Agonist Pilocarpine.” Journal of Neurophysiology 69 (5): 1583–1595.
Shadmehr, Reza, and M A Arbib. 1992. “A Mathematical Analysis of the Force-stiffness
Characteristics of Muscles in Control of a Single Joint System.” Biological Cybernetics 477:
463–477.
Spardy, Lucy E, Sergey N Markin, Natalia A Shevtsova, Boris I Prilutsky, Ilya A Rybak, and
Jonathan E Rubin. 2011. “A Dynamical Systems Analysis of Afferent Control in a
Neuromechanical Model of Locomotion: I. Rhythm Generation.” Journal of Neural
Engineering 8 (6) (December): 065003.
Trappenberg, T. 2009. “Fundamentals of Computational Neuroscience 2e.”
von Twickel, Arndt, Ansgar Büschges, and Frank Pasemann. 2011. “Deriving Neural Network
Controllers from Neuro-biological Data: Implementation of a Single-leg Stick Insect
Controller.” Biological Cybernetics 104 (1-2) (February): 95–119.
von Twickel, Arndt, Manfred Hild, Torsten Siedel, Vishal Patel, and Frank Pasemann. 2011.
“Neural Control of a Modular Multi-legged Walking Machine: Simulation and Hardware.”
Robotics and Autonomous Systems 60 (2) (November): 227–241.
Tóth, Tibor I, S Knops, and Silvia Daun-Gruhn. 2012. “A Neuromechanical Model Explaining
Forward and Backward Stepping in the Stick Insect.” Journal of Neurophysiology 107 (12)
(June): 3267–80.
Watson, James T, and Roy E Ritzmann. 1998. “Leg Kinematics and Muscle Activity During
Treadmill Running in the Cockroach, Blaberus Discoidalis: II. Fast Running.” Journal of
Comparative Physiology. A, Sensory, Neural, and Behavioral Physiology 182 (1) (January):
23–33.
118
Watson, James T, Roy E Ritzmann, and Alan J Pollack. 2002. “Control of Climbing Behavior in
the Cockroach, Blaberus Discoidalis. II. Motor Activities Associated with Joint Movement.”
Journal of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral
Physiology 188 (1) (March): 55–69.
Zill, Sasha N, Ansgar Büschges, and Josef Schmitz. 2011. “Encoding of Force Increases and
Decreases by Tibial Campaniform Sensilla in the Stick Insect, Carausius Morosus.” Journal
of Comparative Physiology. A, Neuroethology, Sensory, Neural, and Behavioral Physiology
197 (8) (August): 851–67.
Zill, Sasha N, S Chaudhry, Ansgar Büschges, and J Schmitz. 2011. “Discrete Sensory Encoding
of Force Increases or Decreases in the Stick Insect Leg.” Society for Neuroscience
(Abstract).
Zill, Sasha N, Bridget R Keller, and Elizabeth R Duke. 2009. “Sensory Signals of Unloading in
One Leg Follow Stance Onset in Another Leg: Transfer of Load and Emergent Coordination
in Cockroach Walking.” Journal of Neurophysiology.
Zill, Sasha N, Josef Schmitz, and Ansgar Büschges. 2004. “Load Sensing and Control of Posture
and Locomotion.” Arthropod Structure & Development 33 (3) (July): 273–86.
