Engineers Guide to PCI Express Solutions

transcript

8/10/2019 Engineers Guide to PCI Express Solutions

1/36

www.eecatalog.com/pcie

PCI ExpressMoves Outsidethe Box

w w.eecatal

Gold Sponsors

Switching SimplifiesPCI Express Testing

Bridging LegacyEquipment with PCIe

PCI-SIG-nificantChanges Brewingin Mobile and Small

form Factor

Switching SimplifiesPCI Express Testing

Bridging LegacyEquipment with PCIe

PCI-SIG-nificantChanges Brewingin Mobile and Small

form Factor



Engineers Guide to

PCI Express Solutions

Engineers Guide to

PCI Express Solutions


2/36

XDS PCIe HMP Line Interface Boards

Designed to Help You Easily Build Your Applications!

Find out more today!

XDS HMP PCIe 4-Port

2/4-Wire E&M Board

Web: xds.amtelco.comE-mail: xds@amtelco.com

(608)838-419

4800 Curtin Drive McFarland, WI 5355

Other H.100 PCIe boards available from AMTELCO XDS:

12/24-Port Loop Start boards 24-Port Station/DID boards

4/8-Span T1/PRI boards 4/8-Span E1/PRA boards 8-Port E&M boards

Announcing the XDS PCIe HMP Modular Line Interface board!

Includes 16 ports of mix-and-match FXO or FXS line interface moduleson a single low-prole PCIe board!NEW

!

Asterisk compliant channel drivers available for use with all AMTELCO PCIe boards.


3/36

Contact MindShare

phone: +1.800.633.1440email: training@mindshare.com

web: www.mindshare.com

PCI Express Technology 3.0 Book and eBook

Authors: Mike Jackson, Ravi BudrukPublisher: MindShare Press

Pages: 1056Retail Price: $94.99

Book Sale Price: $79.99eBook Price: $40.00

Book + eBook Price: $99.99

PCI Express Technology 3.0 Book and eBook

Authors: Mike Jackson, Ravi BudrukPublisher: MindShare Press

Pages: 1056Retail Price:

Book Sale Price: $79.99eBook Price: $40.00

Book + eBook Price: $99.99

.

Classroom/Virtual-Classroom Training and eLearning Courses:

1)

PCI Express 3.0 (5-day course)2)

NVM Express (1-day course)

3)

Mobile PCI Express (M-PCIe) (4-day course)

Classroom/Virtual-Classroom Training and eLearning Courses:

1) PCI Express 3.0 (5-day course)2) NVM Express (1-day course)

3) obile PCI Express (M-PCIe) (4-day course)


4/36Engineers Guide to PCI Express Solutions 2014

Welcome to the 2014Engineers Guide to

PCI Express SolutionsAs Extension Medias Chris Ciufo says, the PCI-SIG is one industry

consortium that doesnt let the grass grow under its feet. Chris took adeep dive into the PCI-SIGs most recent PCI Express announcements

(made earlier this year), and looks ahead as well. You wont want to miss

his insight in PCI-SIG nificant Changes Brewing in Mobile where he

covers everything from the PCIe 3.1 spec through expectations for 4.0, as

well as the M.2 and M-PCIe specifications that are aimed squarely at the

exploding mobile markets. We also looked to our industry experts to get

their take on these changes and more, in our roundtable discussion, PCI

Express Moves Outside the Box.

In other articles, Pericoms Rakesh Bhatia explains how PCIe supports

connectivity with legacy equipment built on interfaces that are all butobsolete in Bridging Legacy Equipment with PCIe. Agilents Rick Eads

addresses the complex task of testing PCI Express transmissionsand

how that will only get tougher with the upcoming Gen 4 release. In his

article, Switching Simplifies PCI Express Testing, he explains how

switches can not only speed up measurements, but can also provide

results that compare favorably or even exceed current test practices. And

because USB 3.0 and PCIe share some common specifications, weve

included Eric Huangs (Synopsys) article, The Accelerating Demand for

10 Gbps SuperSpeed USB 3.0.

Elsewhere in this issue, youll find more information on how this popularstandard takes high-performance into a range of embedded applications.

CESs Akos Csilling looks at the advantages of PCI Express for traditional

VME-based applications in Modern Technology in a Traditional Form

Factor, while Emersons Brian Carr explains how a PCI Express media

processing accelerator card offers benefits for high-density voice and

video processing in Accelerate Server-based Media Processing. Finally,

GEs Peter Thompson explains how multiprocessingand PCI Express

are at the heart of high-performance embedded computing in mil/aero

systems with significantly improved performance in Multiple Processors

in High-Performance Embedded Computing: Only Connect.

Theres all this, plus product information, news, white papers and more,

so dig in and enjoy!

Cheryl Berglund CoupEditor

P.S. To subscribe to our series of Engineers Guides fordevelopers, engineers, designers, and managers, visit:

www.eecatalog.com/pcie

Engineers Guide toPCI ExpressSolutions 20www.eecatalog.com/pcie

Vice President & Publisher

Clair Brightcbright@extensionmedia.com

(415) 255-0390 ext. 15

EditorialVice President/Chief Content OfficerJohn Blyler

jblyler@extensionmedia.com

(503) 614-1082

Editor-in-ChiefChris A. Ciufo

cciufo@extensionmedia.com

EditorCheryl Berglund Coup

ccoupe@extensionmedia.com

Creative/ProductionProduction ManagerSpryte Heithecker

Media CoordinatorJenn Burkhardt

Graphic DesignersNicky Jacobson

Jacob Ewing

Senior Web DeveloperMariam Moattari

Advertising/Reprint SalesVice President & PublisherEmbedded Electronics Media Group

Clair Bright

cbright@extensionmedia.com

(415) 255-0390 ext. 15

Sales ManagerMichael Cloward

mcloward@extensionmedia.com

(415) 255-0390 ext. 17

Marketing/CirculationJenna Johnson

To Subscribewww.eecatalog.com/subscribe

Extension Media, LLCCorporate Office(415)255-0390

President and Group PublisherVince Ridley

vridley@extensionmedia.com

Vice President, SalesEmbedded Electronics Media Group

Clair Bright

cbright@extensionmedia.com

Vice President, Business Development

Melissa Sterlingmsterling@extensionmedia.com

(415)-970-19100

Special Thanks to Our Sponsors

Engineers Guide to PCI Express Solutions is published by Extension Media LLC.Extension Media makes no warranty for the use of its products and assumes noresponsibility for any errors which may appear in this Catalog nor does it make acommitment to update the information contained herein. The Engineers Guide toPCI Express Solutions is Copyright 2013 Extension Media LLC. No informationin this Catalog may be reproduced without expressed written permission fromExtension Media @ 1786 18th Street, San Francisco, CA 94107-2343.All registered trademarks and trademarks included in this Catalog areheld by their respective companies. Every attempt was made to include alltrademarks and registered trademarks where indicated by their companies.


5/36



ContentsPCI Express Moves Outside the Box

By Cheryl Coup, Managing Editor ..........................................................................................................................................5

Bridging Legacy Equipment with PCIe

By Rakesh Bhatia, Pericom .......................................................................................................................................................8

Switching Simplifies PCI Express Testing

By Rick Eads, Agilent Technologies, Inc. ................................................................................................................................ 11

Modern Technology in a Traditional Form Factor

By kos Csilling, CES - Creative Electronic Systems SA ........................................................................................................ 14

Accelerate Server-based Media Processing

By Brian Carr, Emerson Network Power .................................................................................................................................18

PCI-SIG-nificant Changes Brewing in Mobile and Small Form Factor Designs

By Chris A. Ciufo, Editor-in-Chief ............................................................................................................................................21

Multiple Processors in High-Performance Embedded Computing: Only Connect

By Peter Thompson, GE Intelligent Platforms .........................................................................................................................23

The Accelerating Demand for 10 Gbps SuperSpeed USB 3.0

By Eric Huang, Synopsys ........................................................................................................................................................26

PRODUCT SERVICES

Board-to-Board

Boards/Hardware

ACCES I/O Products, Inc.PCI Express Data Acquisition and Control Cards ..........28

General Standards CorporationAnalog, Digital and Serial I/O Boards ...........................29

Chip-to-Chip

ICs

PLX TechnologyExpressLane PCI Express 3.0/2.0/1.x .........................30

ExpressLane PCI Express 3.0 Switches ......................31

Test and Analysis

Teledyne LeCroyTeledyne LeCroys PCI Express ProtocolAnalysis and Test Tools .................................................32


7/36www.eecatalog.com/pcie

SPECIAL FEATURE

Its tough to limit a discussion on PCI Express, as thestandard continues to evolve into nearly every aspect ofcomputing. New generations will support highly demandingapplications in the data center as well as remote exploration,surveillance and research equipment that collect and pro-cess large amounts of data before they can be transported

for the next level of processing. And new efforts are takingthe standard out of the box to cable, mobile and small formfactors. John Wiedemeier, senior product marketing man-ager for eledyne LeCroy and Larry Chisvin, vice presidentof strategic initiatives for PLX echnology were kind enoughto offer their views on these topics and more.

EECatalog: How are the different PCI Express (PCIe) ver-sions being implemented in embedded systems today? Doyou expect that to change?

John Wiedemeier, Teledyne LeCroy: Historically,the embedded market has been more con-

servative to adopt higher speeds into theirsystems. Many of the embedded productsare satisfied with their performance but

looking for competitive differentiators thatcan add more product value. Improving I/O

performance is a sure way to do that. Definitely, weare seeing companies transition from PCIe Gen1 (2.5G/s)to Gen2 (5G/s). Most embedded companies seem contentfor now at PCIe 2.0 and have not moved to the higher-speedPCIe 3.0 standard. New opportunities with PCIe-based

storage and lower pricing on fast PCIe-based CPUs will cer-tainly be game changers in the near future.

Larry Chisvin, PLX Technology: PCI Expresshas been a part of embedded design since

the specification was first released, largelydue to its software compatibility with PCI,which had already permeated the embedded

world. PCIe offers low cost, high performanceand exibility, and since embedded systems

are largely built with the same components used inother, higher-volume markets, PCIe is an ideal way to con-

nect together the components inside embedded systems.As with most of the industry, PCIe Gen 2 has been broadlyadopted, and Gen 3 is showing up in newer systems. Sinceembedded platforms have long lives, designers tend to usecomponents that will not soon be obsolete, but at the sametime the components must have an established track record

of quality; embedded systems almost never crash. So ear-lier generations of PCIe are used where the performance isadequate, and newer versions where the trade-off pushesthe designers toward additional performance.

PCIe has been adopted in many standards developed for theembedded industry, such as PICMG 3.4 for the ACA chassisand VIA 41 & 46 specifications for various connectivityand switch fabric applications. Tis is likely to continue,as each version of PCIe has full backward compatibility ofsoftware and firmware for a seamless and easy migrationexperience. PCIe Gen 3 allows for advanced applications,such as a PCIe switch-based fabric, such as PLXs Express-

Fabric, that allows for unprecedented consolidation,convergence and power-savings in data centers and cloudenvironments, thanks to the reduced need for componentsand board space.

By Cheryl Coup, Managing Editor

PCI Express Moves Outside the BoxOur roundtable discussion hits PCIes generational shifts to ad-

dress demanding requirements, the outlook for the new PCIe OCu-

Link cable, and the move to mobile and low-power applications.

Using PCIe instead of

Ethernet for short-distance,

rack-level communication

can offer substantial cost and

power advantages, and this

will continue to be the case

as PCIe moves to Gen 4.



SPECIAL FEATURE

EECatalog: Where do you anticipate seeing implementa-tions of PCI Express 4.0, which will boast 16 G/s (at 16Gbps) or about 64GB/s (x16)? Will it give Ethernet anycause for concern?

Wiedemeier, Teledyne LeCroy:Te demand for cloud com-puting with its infinite networked virtualized data storageis driving I/O performance. PCI Express 4.0 at 16G/s

will deliver the speed and performance required to meetthese challenges. High-speed interconnects at these datarates will drive faster servers and high-capacity SSD-basedstorage systems.

Ethernet will have its place connecting networks and datacenters to the cloud providing carrier Ethernet-based ser-vices. PCI Express 4.0 with its doubling of performance andlow latency will build the next-generation infrastructure ofnetworked servers and storage systems. Protocol analysis willbecome important as devel-opers work to achieve quality

products and overcome issuesrelated to these big changes inthis new specification.

Chisvin, PLX Technology: Aswith every version of PCIe,the first implementationswill show up in the graphicsand high-end server markets,where either the bandwidth isnecessary or the applicationis able to command a sub-stantial premium. Tis will

be followed quickly by thestorage market, where ultra-high-performance SSD-basedsystems will strain the capabilities of the fabric and back-bone. Over time, Gen 4 will migrate to the general-purposedata center and cloud markets, as the components beingconnected together continue to push the need for greaterinterconnection speeds.

Overall, this will not impact Ethernet in a major way,since there is an infrastructural advantage that Ethernethas developed, and since the Ethernet suppliers are alsopushing their performance higher for the same reasons. In

fact, PCIe and Ethernet are highly complementary of oneanother; they already coexist in a wide range of systemsacross several market segments. Te one area that Ethernetwill be impacted is inside the rack of a data center or cloudprovider, and this has already started. Using PCIe instead ofEthernet for short-distance, rack-level communication canoffer substantial cost and power advantages, and this willcontinue to be the case as PCIe moves to Gen 4.

PCIe Gen 4 is not expected to be deployed in the applica-tions above before the 2016-17 timeframe, but when it doesappear it will be utilized in the same manner as current

PCIe generations, but twice as fast. Applications that couldbenefit from Gen 4s higher speed are future unmannedaerial vehicles, space/atmosphere research and undersea/underground-exploration equipment that need to col-lect and process large amounts of data before they can betransported for the next level of processing. Additionally,unmanned air (ground and underground) vehicles in com-mercial applications will be a prime target for PCIe, both

Gen 3 and Gen 4.

EECatalog:PCI Express is planning for an expansion out-side the box using a low-cost cable that the PCI-SIG says willrival Apple's Tunderbolt for external devices, and can beused as a PCIe 32Gbps simplex (4-lane) bus extender. Whatare your predictions for the PCIe OCuLink cable?

Wiedemeier, Teledyne LeCroy:Tere are tremendous advan-

tages to adopting the PCIeOCuLink cable besides speed.For one, a proprietary con-troller chip is not required toenable this to work with sys-tems that use a PCI ExpressCPU-based system board.

his will allow system ven-dors to provide faster andmore cost-effective prod-ucts to market. However,with all of its merits, if the

PCIe community does notrally around this new standard it will not gain the samevisibility as hunderboldt. I think this cable has a lot ofpotential with storage systems. It wil l provide the neededthroughput to support the new high-capacity SSDs instorage and server systems.

Chisvin, PLX Technology: Tere are attractive reasons fortaking PCIe out of the box, and key among the advantagesis that the software model is vastly simplified when thesystem does not need to worry about where the devicesreside physically. For example, performance suffersand

complexity increaseswhen data communicates on amotherboard using PCIe, then needs to be translated toEthernet for transmission to another box, then translatedback to connect up to all of the components. By keeping thedata in its native PCIe form, the protocol translation over-head is reduced and meaningful performance improvementcan be achieved. If the OcuLink cable can achieve its pricetargets, it should be very successful, since it will providethe ability to offer the advantages outlined. Of course, youcan get most of those same benefits today by using existingPCIe, QSFP or miniSAS-HD cables, running the PCIe pro-tocol across them, though at a higher cost.

With all of its merits, if

the PCIe community does

not rally around this new

standard [PCIe OCuLink]

it will not gain the same

visibility as Thunderboldt.



SPECIAL FEATURE

PCIe OcuLink will offer active, passive and hybrid copperand optical cabling. Te user will be able to select the cablingthat suits the cost targets of the application they are serving.OcuLink, being an open standard, will allow volume manu-facturers to build cables that are competitive in cost.

EECatalog:Te forecasts show the PC market is shrinkingwhether in units delivered, processor forecasts or mostrecently, the number of GPUs forecast. With the PC goesPCI Express, so the PCI-SIG's M-PCIe technology is focusingon mobile markets, including smartphones, tablets andportable embedded. What are the technical challenges (andupsides) to extending PCI Express into mobile?

Wiedemeier, Teledyne LeCroy: Tere are two challenges thathad to be addressed for PCI Express to move successfully intothe mobile market. Te first challenge was to conquer PCIExpresss ability to operate

in low-power designs wherelong battery life is essential.M-PCIe is able to solve thisby replacing the PCIe PHYwith the new MIPI AlliancesM-PHY. Te low-powerM-PHY provides proven powerefficiency to enable low-powerdesigns for mobile.

Te second challenge was how to migrate the existing PCIExpress software infrastructure in current PCI Expressdesigns to the newer power-efficient architecture of M-PCIe.

M-PCIe preserves the same upper protocol layers of PCIe,making it possible to do this. Maintaining the link andtransaction layers as in the old PCIe programming modelallows legacy software structure to easily port to the newM-PCIe programming model. eledyne LeCroy has recentlyprovided for free a new software utility for M-PCIe applica-tions that will help developers view and edit their M-PCIedevices configuration space registers. Providing tools tothis new industry will help insure a successful transitionfor developers to this low-power architecture.

Chisvin, PLX Technology:I would challenge the belief that

the success of PCIe is dependent on the PC market. In gen-eral, PCs tend to offer a limited opportunity for PCIe vendors;they simply connect up these highly integrated devices atlimited performance levels. PCIe has already expanded wellbeyond that and is used in high-performance systems in thestorage, server, communications and embedded markets. Infact, if the primary purpose of PCIe was just for PCs, therewould be no reason to be extending it to Gen 4. PCIe isnow being driven by the broad market need for a standard,exible, low-cost, high-performance interconnect, and hasestablished itself as the primary general-purpose mecha-nism for this requirement.

Tat being said, expanding the total available market isalways a good thing. Te main challenge to using PCIe inthe mobile market has been power, and that is what is beingaddressed by the PCI-SIG with the introduction of numerousenhancements to the protocol. By offering a power-efficientway to use PCIe in mobile devices, a new and large marketopportunity has been created.

EECatalog:Small form factors are all the rage, in avors rangingfrom PCIe/104 and AdvancedMC, to Gumstix and COMExpress. Te PCI-SIG's M.2 spec is targeting ultra-light (thinkUltrabooks) and ultra-thin (think tablets) devices. What areyour thoughts on this new mobile-focused form factor?

Wiedemeier, Teledyne LeCroy: Te new M.2 form factoris a great way to use SSD technology in thin-computingappliances where size and connectivity are restrained. M.2

devices will start out with

Socket 2 (SAA Express x2)based devices and eventu-ally move to Socket 3 higherperformance M.2 deviceslater. From a test equipmentperspective, eledyne LeCroyis highly involved withproviding ways to test newdevices like this. Recently, weintroduced an interposer for

our PCIe protocol analyzer to support testing M.2 devices.Socket 2 and Socket 3 type M.2 devices which support SAAExpress or NVMe can be connected to and analyzed for pro-

tocol correctness and performance.

Chisvin, PLX Technology: As is the case with the M-PCIeprotocol, extending PCIes ability to fit into highly denseapplications in a standard way enables the technology tobe used in places that had previously been excluded. Sincealmost every component already connects to its neighborsthrough PCIe, the new connector standard matches wellwith device packaging technology that has allowed highlycompact consumer devices. In the case of M.2, one of themost exciting usage models will be high-performance, low-power, ash memory-based storage that eliminates the

need for the rotating media, thus dramatically reducingpower needs and extending battery life.

Cheryl Berglund Coup is editor of EECatalog.com. Her articles have appeared in EE imes,

Electronic Business, Microsoft Embedded Re-view and Windows Developers Journal and shehas developed presentations for the EmbeddedSystems Conference and ICSPA. She has held avariety of production, technical marketing and writing positionswithin technology companies and agencies in the Northwest.

I would challenge the belief

that the success of PCIe is

dependent on the PC market.



SPECIAL FEATURE

For many applications, product life is measured in yearsor decades. Tis trend is common among industrial andmedical applications, as well as any industry where productlongevity is expected. As a result, a technology disconnectcan arise between long-life equipment and the tools used tointerface to or control them.

For example, a machining center deployed in the 90sand still in operation today is most likely outfitted witha UAR-based port such as RS-232 for data uploadingand diagnostics. However, PCs and laptops no longership with an available RS-232 port. o continue to sup-port legacy equipment with new software and diagnostictools, OEMs need to be able connect equipment usingcurrent technology.

Note that this disconnect continues to be an issue even asequipment evolves. In many cases, the UAR interface wasreplaced with a newer interface such as USB or PCI, com-

monly known as conventional PCI today. PCI in turn hasbeen replaced by PCI Express (PCIe), so even PCI-basedequipment can be difficult to connect to with current com-puting platforms. Tis article will address how OEMs acrossindustries can use PCIe to support connectivity with legacyequipment built upon interfaces like RS-232 and PCI thatare obsolete in the consumer electronics world.

Figure 1a and 1b show two basic architectures for a legacybridge. Te UAR output from the system or piece of equip-ment is routed to a bridge, bridged to PCIe and then routedto the control point, typically a PC, laptop or diagnostic tool.

Te bridge can either be a standalone box that connects tothe control point via a cable or be integrated onto a PCIecard that plugs directly into the control point.

oday, bridging circuitry is avai lable as an integrated com-ponent. A robust legacy bridge, however, requires morethan simple conversion between interconnects. Troughthe use of switching technology, the exibility and effi-ciency of a legacy bridge can be significantly improved,leading to overall lower operating costs and simplifiedsystem management. Signal conditioning can also beintroduced to improve signal integrity and ensure systemreliability under all operating conditions. Finally, a low-jitter timing source can ensure increased signal margin toease design complexity.

Flexibility and Efficiency

Multiplexing enables two common ways to increase theexibility and efficiency of an adapter bridge: throughconsolidating input ports and by supporting multipleoutput types. Consider applications such as in a hospitalor on a factory oor, where several pieces of equipmentare being monitored and/or controlled. Rather than havea separate converter for each piece, a single bridge canservice multiple stations.

Given that PCIe offers substantially greater data ratesthan UAR-based interconnects, one PCIe stream can

By Rakesh Bhatia, Pericom

Bridging LegacyEquipment with PCIePCIe supports connectivity with legacy equipment built on interfaces such

as RS-232 and PCI that are obsolete in the consumer electronics world.

Even with a PCIe plug-in

card that does not require

a cable, signal integrity can

still be an issue.

Figure 1a and 1b: The UART output is bridged to PCIe, and then routedto the control point, typically a PC, laptop, or diagnostic tool. The bridgecan either be a standalone box (a) or integrated onto a PCIe card (b).



SPECIAL FEATURE

easily accommodate several UAR streams. Integration at

the IC level enables a single multi-port bridge chip to per-form the work of several individual bridges. For example,in Figure 2, a 4-port UAR-to-PCIe bridge reduces thedesign footprint.

Consolidation offers manycost benefits. Rather thanhaving to purchase multipleconvertors, technicians onlyrequire a single convertor tosupport several end points. Asa consequence, fewer control

points are required as well.Tis can result in substantialcost savings, especially if aseparate PC/laptop is used for each station (see Figure 3).Consolidation also simplifies management of larger banksof equipment by reducing the number of control pointsrequired. Since there are fewer points of potential failure,reliability improves as well.

Supporting multiple protocols through switching can also

increase the exibility of a bridge by enabling it to supportmultiple interconnects on the output side. For example, bysupporting both PCIe and PCI,a bridge can work not onlywith new PCs and laptops butalso older control points thatonly support PCI. Dependingupon the compute capabilitiesrequired by the control point,this gives system architectsthe exibility to use olderequipment that is availablefor use rather than have to

purchase new equipment forthis task. In addition, it pro-vides a seamless transition

when a PCI-based control point finally fails and needs to bereplaced with a PCIe-based platform.

Dual interconnect support can be implemented witha switch and bridge (see Figure 4). he switch controlswhether the output port is connected directly to the PCIe

output port or through a PCIe-to-PCI bridge.his approach allows additional PCIe endpointsto be connected to a single host controller.

Signal Integrity

Te issues affecting reliability are vastly differentbetween a low-speed interconnect and one basedon high-frequency signals. Even at its highest datarates, an RS-232 interconnect could be run 300meters without causing reliability concerns. At thehigher frequency of PCIe, however, signal integritybecomes more of a concern because of its greatlysusceptibility to noise.

A redriver is a bi-directional

signal conditioner that

uses equalization and pre-

emphasis to compensate for

known losses.

Figure 2: Consolidating multiple UART ports to a single output simpli-fies adapter design.

Figure 3: A single bridge can support multiple end points to reduce theoverall number of control points required.

Figure 4: A bridge can automatically switch between PCIe and PCI for seamless con-nectivity to different types of control points.



SPECIAL FEATURE

For example, if the converter is a standalone box, data willneed to run over a combination of a cable, multiple con-nectors, PCB traces and vias, all of which degrade the PCIesignal. Even with a PCIe plug-in card that does not require acable, signal integrity can still be an issue, depending uponhow far into the PC or laptop the PCIe controller is located,whether low-cost components were used in the signal pathand how carefully the traces were laid out.

A clean signal on the receiver side will appear to have whatis referred to as an open eye. As signal losses increase, thesignal eye will close, as shown in Figure 5a. Performance

and reliability of the interconnect will degrade as well. Notethat even though the PCIe interconnect may be carryingrelatively little data, it is still sending it at 2.5 Gbps, sosignal integrity needs to be considered. Most common inembedded platforms today, PCIe 2.0 5 Gb definitely pres-ents a signal-integrity challenge to designers. Te typicalembedded CPU has limited drive capability, which shortensthe PCB trace distances. Compounded with signal path con-nectors, vias and even trace pitch distances, a redriver orrepeater becomes necessary in many platforms.

o open the signal eye and restore signal integrity, a redriver

or repeater can be placed in the signal chain. A redriver isa bi-directional signal conditioner that uses equalizationand pre-emphasis to compensate for known losses. Onthe transmit side, the redriver boosts the signal to closelymatch losses. Te signal still experiences losses as it travelsto the receiver, but instead of the signal arriving with aclosed eye, the signal more closely resembles the originalsignal (see Figure 5b). Similarly, a redriver can also restorethe signal quality of received signals by compensating forknown losses along the signal path (see Figure 5c).

Another aspect of signal integrity to consider is the accu-racy of the timing source. Because PCIe signal frequencyis dependent upon the system clock, any jitter and driftintroduced by the clock degrades signal integrity. Utilizinga low-jitter timing source eliminates these concerns andmaximizes signal integrity from its source on.

Increasing signal integrity can simplify the design anduse of a converter in several ways. First, signals can besent reliably over greater distances. Tis means longer,more convenient cables can be used, and the signal cantravel over longer PCB traces. On the converter side, this

gives developers greater signal margin to work with, eitherto relax other design constraints or use less-expensivecomponents. Alternatively, the bridge can compensate forpotential signal losses in the receiving control point.

Maintaining compatibility of legacy equipment to moderndiagnostic equipment enables system architects to continueto maximize utility over the equipments operating life. Byemploying switching, signal conditioning and low-jittertiming technology, OEMs can provide exibility, efficiency,and reliability while lowering operating costs and easingmanagement complexity.

Rakesh Bhatia is a product marketing directorfor PCIe products at Pericom and has over 15years of semiconductor industry experience.Rakesh has published several articles andwhite papers. He has a B.S. in electrical en-

gineering from University of Houston and anMBA from California State University.

Figure 5: As signal losses increase, the signal eye will close (a), degrading performance and reliability. Signal conditioning through a redriver orrepeater compensates for known losses at the transmitter (b) and receiver (c) to open the eye and restore signal quality.



SPECIAL FEATURE

PCI Express (PCIe) is a multi-line, high-speed serial businterface that can operate at up to 8 gigabits per second perlane of traffic with up to 16 lanes of communication utilizedbetween transmitter and receiver. One lane designates botha differential transmit and receive pair of conductors. PCIExpress is owned by a non-profit organization called the

PCI Special Interest Group (PCI-SIG) which consists of justunder 800 member companies. Te PCI-SIG announced lastyear that the next generation of the standard, PCI Express4.0, will double the data rate compared to the current gen-eration, achieving 16 gigabits per second. With each newgeneration, the task involved in validating the electricalperformance of an integrated circuit, endpoint (add-in card)or root complex (system) device becomes more complicated.

Faster Data Rates Complicate Testing

When the PCIe standard came on the scene just over 10 yearsago it operated at a single speed of 2.5 gigabits per second.esting this technology was straightforward as a single cap-

ture of each lane of the transmitter was all that was neededto perform an analysis of transmitter characteristics such aseye opening, jitter and voltage levels. Now the PCI-SIG hasadded two additional data rates (5 gigabits/s and 8 gigabits/s)and additional levels of de-emphasis for each of those newspeeds. De-emphasis (or pre-emphasis as it is sometimescalled in other standards) is a transmitter-oriented typeof equalization. Te non-transition bits in a transmissionsequence are de-emphasized electrically such that thechange in electrical levels between the transition and non-transition bits compensates somewhat for the inverse of thestep response of a given channel. Ideally, the signal amplitude

of both transition and non-transition bits should be roughlythe same when measured at the end of a channel.

For PCIe 1.0, only one de-emphasis level was used. ForPCIe 2.0, two different levels of de-emphasis are speci-fied for the transmitter (-3.5dB and -6dB). Generally, themore de-emphasis at 5 GB/s, the longer the channel usedin the system. PCIe is designed to support two differentchannel lengths. A short (or client) channel is about 10inches (25cm) and can have up to one connector. Te PCIestandard also supports a longer 20-inch (50cm) channelwith two connectors. Te shorter channel tends to be more

reection-dominated and the longer channel tends to bemore loss-dominated. For PCIe 3.0, each of these channeltopologies are also supported; however, at 8 GBit/s, thereis a significant amount of loss at the nyquist frequency of4GHz so that transmitter-based equalization was insuf-ficient to guarantee that a long 20-inch channel would be

a stable and interoperable architecture. Tus, for PCIe 3.0the de-emphasis space was significantly expanded and areceiver equalization requirement was also added. For sim-plicity, the PCIe 3.0 standard requires a transmitter to drive11 different de-emphasis levels at 8 GBit/s. Tese 11 dif-ferent levels are designated as presets and are numberedP0 through P10.

Test Requirements for Current Generations

Te testing of all current generations of PCI Express fromPCIe 1.0 to PCIe 2.0 to PCIe 3.0 requires testing the outputof a transmitter not only at the different supported speedsof 2.5GBit/s, 5GBit/s and 8GBit/s, but it also requires that

each de-emphasis level be tested at each supported speed.In the worst-case scenario it is necessary to test 14 differentsignal conditions for each lane supported by a device. Forexample, with a 16-lane PCIe 3.0 device it would be neces-sary to test 14 signals multiplied by 16 lanes which equals atotal of 224 signals!

In the case of both add-in cards and motherboards,capturing signals usually involves dedicated PCIe test fix-tures (that can be obtained from the PCI-SIG), test cables,adapters and a real-time oscilloscope. Te device under test(DU) is placed in either a compliance load board for testing

a root-complex device, or compliance base board for an end-point device. Te PCIe DU can automatically generate atest signal and can be toggled to each of the three data ratesusing a specialized function integrated into each standardtest fixtures. A typical test session follows these steps:

By Rick Eads, Agilent Technologies, Inc.

Switching Simplifies PCIExpress TestingSwitches can not only speed up your measurements but can also give you

results that compare favorably or even exceed current test practices.



SPECIAL FEATURE

voltage levels)

If the DU is a 16-lane device, it would be necessary to movecables from lane to lane until all lanes are tested. In additionto 50 ohm terminators must be inserted into each unused lane

that is not being tested. As cables are moved from lane to lane,the terminators must also be moved. Since the CLB and CBBfixtures used for PCIe 3.0 testing use small SMP connectors,the small SMP terminators must be carefully removed and re-inserted with care to not damage or bend the center pin.

PCI Express 4.0 Will Add Even More Complexity

Keep in mind that PCIe 4.0 is in development and it will addsignificantly to an already complex and cumbersome test pro-cess. At 16 GBit/s, PCIe 4.0 will add another 11 de-emphasispresets, each operating at the new higher rate. Tis meansthat a PCIe 4.0 device having 16 lanes will require the testingof 25 signals per lane for a total of 400 test signals that must

each be configured, captured and analyzed.

In order to alleviate some of this complexity, it is possibleto use high-frequency, high-quality switch networks to per-form the task of multiplexing the lanes of a PCIe device intothe limited (two or four) channels of a real-time oscil loscope.Tere are a variety of microwave switch networks availablein multiple configurations. Tese switches can support sixdifferential lanes into a single differential output. Tere areother switches that can support 16 differential lanes intotwo differential outputs. Also, multiple switches can be cas-caded together in order to expand the capacity of a single

switch to support as many lanes as might be desired.

Pros and Consof High-Frequency Switching

Te biggest advantage of using a high-frequency switchas part of the setup for testing a multi-lane PCIe switchsignificantly decreased test times. It is possible to cut youroverall test time by 50% when using a switch networkbecause it is no longer necessary to manually change cableconfigurations on the test fixture. Other advantages includethe elimination of connection errors. Once the setup hasbeen properly configured and verified, it doesnt have to betouched again so there is no risk of not seating a SMP cable

properly into the connector. est fixtures gain increased ser-vice life since you eliminate the multiple insertion/removalcycles and this lowers the risk that a test fixture might beaccidentally damaged. Lastly, there is the opportunity tohave an automated control program oversee both the teststhat are executed and the automatic switching of the lanesbeing tested. Switches thus offer the promise of significant

time savings through programmatic test automation, whichcan both speed up the completion of tests and increase thequality and consistency of test results from DU to DU.

Te use of switches does, however, have some disadvantages.For example, even though microwave switches typically have ananalog bandwidth of 26 GHz, they still are not completely loss-less. Using switches for multi-lane testing also requires multiplecables which add attenuation to the signals being analyzed. Andfinally, switches are not free. Because they offer a high-qualityconnection between the DU and the instrumentation, theycan cost $5K-$15K depending on the configuration used.

One way to overcome the additional loss of adding a switch net-work is to measure the loss of each channel of the switch that youare using along with any additional cables. Once you have mea-sured each channel, it is then possible to apply a filter within theoscilloscope that will compensate for the loss of the switch andthe cables and connectors.

Figure 2: An Agilent E5071C 20GHz ENANetwork Analzyer is used to calculate thefrequency-dependent performance of thecombined cable, SMA adapter and microwaveswitch network.

Figure 1: An Agilent U3020A S26 26.5GHzswitch to multiplexes multi-lane PCIExpress signals into a high-performancereal-time oscilloscope.

Figure 4: Using the U3020A switch to measure 5 lanes of PCI Express8GT/s signals, this chart shows that using a switch with either VNA- orTDT-based compensation actually measures larger eyes compared tomeasurements performed without either a switch + compensation.

Figure 3: The Agilent DSO-X 93204A 32GHzoscilloscope supports an optional TDTanalysis tool that compares to a VNA-basedapproach to measuring frequency-depen-dent characteristics.



SPECIAL FEATURE

Rick Eads is a principal product manager atAgilent echnologies. Rick has served on theBoard of Directors for the PCI-SIG since 2007,contributed to PCIe 3.0 BASE, CEM, test

specification development and has led electri-cal Gold Suite testing at PCI-SIG-sponsoredworkshops worldwide.

Tere are a number of tools which can be used to measureeach signal path through the switch. One way is to measureeach path using a vector network analyzer (or VNA). TeVNA will capture the frequency-dependent loss charac-teristics of each signal path, which you can then load intoyour oscilloscope and, with the appropriate hardware (orsoftware option), create a filter which exactly compensatesfor the specific loss profile of each signal path through the

switch network. As long as the signal path has less thanabout -10dB of loss at the highest frequency you choose tomeasure, you should be able to compensate for the switchaccurately without also amplifying asignificant amount of random noise.

Not all labs have access to a VNA sothere are other options for measuringthe frequency-dependent loss charac-teristics of a switch. Te Agilent 90000X-series oscilloscopes for example,have an option called Precision Cable

which allows you to characterize switchand cable characteristics up to about36GHz using an approach similar to aD. Likewise, if you have access to aDR, you can also use it to measure thefrequency-domain performance of theswitch and cable paths.

In Figure 4, you can see two differentapproaches to using frequency-domainanalysis of signal-loss characteristics tocompensate for cable and switch losses.What is interesting is that eye-height

measurements for an 8GBit/s signalthrough a compensated switch wereabout 15-20% larger compared with usinga simple, direct-cabled connection (noswitch) to the DU.

In conclusion, switching is the onlyway to greatly simplify the testing ofmulti-speed, multi-lane serial bussesby eliminating the need to change con-nections. Nevertheless, using switchesin a test setup also demands the use

of signal-path measurement and toolsand techniques to compensate for theadditional loss resulting from the useof the switch and additional cables.Using the right equipment, switchescan not only speed up your measure-ments but can also give you resultsthat compare favorably or even exceedcurrent test practices.



SPECIAL FEATURE

raditional VME form factors are more and more relegatedto legacy status, with most new high-end systems beingimplemented in VPX. Yet not all applications require hugebandwidths or a large number of high-speed connections.In many cases the previous investment in the enclosuresand custom boards fully justifies the choice to stick to theVME format. On the other hand, the traditional VMEand PCI buses are often unable to meet the current datatransfer requirements. In addition, the industry-standard

single-board computers often rely on single-source interfacecomponents, which may also present obsolescence issues.

Many avionic computers require multiple avionic interfaces,some video or signal processing capability and onboard datastorage. Tese types of computers provide important, butnot critical functions (Design Assurance Level DAL-C or Dper FAA guidelines). Te console and Ethernet interfaces areoften used for maintenance and non-critical communication.Te entire system must be rugged to resist the environmentalconditions of the aircraft, with extreme temperatures, vibra-tion, EMI/EMC and various contaminants.

In the past, this would require multiple single-board com-puters in VME or CPCI format, each providing a subset ofthe functions. Te limited interconnect bandwidth wouldlimit the performance and require careful partitioningof the application in order to limit the data-exchangerequirements. ime synchronization among the SBCswould be critical, in order to ensure coherent operation ofthe various interfaces.

oday, a single processor, possibly with multiple cores, cando the job. One core can be dedicated to time-critical func-tions, while the other can offload non-critical activity. Onthe other hand, the connection to the multiple I/O inter-faces requires more bandwidth than a traditional VME orPCI bus. PCI Express can provide this additional bandwidthin a simple and software-compatible way. PCIe is readilyavailable in modern processors and FPGA devices. A singleGen 1.0 PCIe x4 interface can transfer 2 GB/s. Most func-tions are available as XMC modules that can be directlyplugged onto the SBC, or onto a suitable carrier.

By kos Csilling, CES - Creative Electronic Systems SA

Modern Technology in aTraditional Form FactorNew products combine the advantages of modern processor and interconnect

technology such as PCI Express with the benefits of the traditional VME form factor.

Figure 1: Block diagram of a typical avionic computer platform. A single modern CPU, with an FPGA for simple I/O customization, extended with asolid-state storage device, a PMC or XMC for avionic interfaces and a video-processing mezzanine.



SPECIAL FEATURE

pression engine were installed on this carrier, completingthe functionality.

A sealed, conduction-cooled enclosure was already avail-able, with an integrated VME64x backplane and powersupply. Te external connectors and the electrical protec-tion devices were installed on a separate protection board,linked to the overlay backplane through a exible PCBconnection, with an additional board for the electrical filterand the hold-up capacitors.

Te resulting design provides new technology with theFreescale QorIQ processor and the PCIe interconnect inthe proven mechanical and electrical environment of aVME64x system.

kos Csilling is a product development man-ager at CES - Creative Electronic Systems SA.

He works with the CO on the product strategyand roadmap. Previously he managed the de-velopment of VME-based and custom avionic

computer platforms. Before he joined CES in2003, he was a post-doctoral fellow at CERN, the EuropeanOrganization for Nuclear Research. He holds an MSc and a

PhD in Physics from Eotvos University in Budapest and ispursuing an executive MBA at HEC Geneva.

Te VIA 41 (VXS) format replaces the VME-P0 connectorwith a more powerful one that allows the transmission ofeight pairs of high-speed signals, up to 10Gb/s; for example

PCIe, SRIO or any other serial protocol. Te standard allowsthe signals to be routed in multiple topologies, for examplein a daisy chain to adjacent slots, or in a star or dual-star toone or two central switch slots, for larger systems. WhileVXS does definitely increase the bandwidth available, anddoes allow the coexistence of new and old boards, it alsorequires a new backplane. Still, VXS is successful in anumber of applications where it allows a gradual transitionfrom VME or CPCI to a switched serial fabric.

Te PCIe over P0 technology introduced recently by somevendors extends this possibility to the traditional VMEform factor. Tis new technology allows the continued use

of the existing VME64x systems, including the backplanesand legacy boards, to achieve up to 2 GB/s point-to-pointtransfer rates. Te technology is based on a special P0 con-nector on the board in order to provide extended signalintegrity to allow the establishment of PCIe links betweentwo or three VME slots over the traditional VME-P0 con-nector of the backplane. Tese signals are not routed in thebackplane, but in a custom overlay, allowing any intercon-nect topology. Tis technology provides a cost-effectivesolution to increase data transfer rates between a smallnumber of processors in any VME64x systems, without theneed to replace the existing backplane.

Recently, for the design of a new ight computer with mul-tiple avionic interfaces including discrete signals and serialports, input video channels and onboard data storage, CESdecided to use a traditional VME format. Te RIO6-8096was chosen as the main processing element, with a storageXMC installed directly on this SBC. Te optional FPGA onthe RIO6 was used to provide discrete and serial I/O. Teelectrical interfaces for these signals were installed onan overlay backplane, which also provided the PCIe linkthrough the VME-P0 to a switching carrier installed in asecond VME slot. An avionic I/O module and a video com-

Figure 3: This avionic computer is based on the RIO6-8096, and providmultiple avionic interfaces, including MIL-1553, ARINC-429, RS-422 andiscrete signals. It also provides two video inputs with compressioand storage capability. The sealed conduction-cooled enclosure ca

withstand the severe environmental conditions encountered in aborne applications.

Figure 2: The RIO6-8096 SBC from CES is based on the FreescaleQorIQ P2020 processor. It provides up to x8 PCIe link over theVME-P0 connector, a VME interface fully implemented in the FPGA,an optional user-programmable FPGA, and FlexIO technology foreasy pin-out customization.


18/36

Copyright 2013 RTD Embedded Technologies, Inc. All rights reserved. All trademarks or registered trademarks are the property of their respective companies.

RTD Designs and Manufactures a Complete Line of High-Reliability Embedded Products & Accessories

AS9100 and ISO9001 Certied

PCI Express, PCI, and ISA Experts

Plus


19/36

The products above are just a sampling of RTDsboard-level and ruggedized packaging solutions. Visitwww.rtd.com to see our complete product list.

AS9100 & ISO9001 Certied



SPECIAL FEATURE

Tese go to eleven, were the legendary words of Spinal aplead guitarist Nigel ufnel when explaining why the bandsamplifiers made them louder, and therefore better, thanother rock bands. Te expression now has its own entry inthe Oxford English Dictionary and has come to mean theact of taking something to the extreme. When talking about

communications networks, taking the performance of anapplication to eleven usually means adding more equipment.Tis is especially true when adding functionality withoutcompromising performance of the original applications onequipment such as rack mounted servers and appliances.

For example, high-density voice and video processing isincreasingly in demand for network applications and the cur-rent typical solution is some kind of commercial host mediaprocessing (HMP) solution that draws on the existing pro-cessor resources of the server.

Tis article describes an alternative approach, using PCI

Express media acceleration cardswhich are increasinglyavailable off the shelfand embedded voice/video firmwareto offer dramatically improved performance while taking upless space, consuming less power and costing less. In short, asolution that can take a server-based media processing appli-cation to eleven.

As communications networks transition to an all-IP environ-ment, service providers and network operators are finding aneed for IP media servers and new advanced ow manage-ment devices such as session border controllers, QOS analyticengines and intelligent ow optimizers.

Many of these are developed and deployed on 1U or 2U stan-dard rack-mounted server architectures for simplicity. Terole of IP media gateways and media servers is clear but asthe developers and users of border ow management devicesconsider where to go next, one obvious step is to build someadvanced media stream processing into the platform. Onekey concern is scalability. According to most analysts, mobiledata and especially mobile video is expected to grow expo-nentially over the next three to five years so the pressure ison to find cost and power-efficient ways to scale media pro-

cessing to suit. Some of the issues that confront equipmentdevelopers are as follows:

Adding Voice Transcoding

to a Session Border Controller

A good example of a ow management application is thesession border controller (SBC), an often-quoted example ofa class of equipment known as network security gateways.Tese are characteristic of bump in the wire devices thatform a bridge between trusted and untrusted networks orenterprises. Teir job is to analyze and characterize incomingIP traffic, block undesirable or unauthorized ows and let

through approved traffic. In communications networks, a lotof this traffic is media streams.

As this is a gateway point, many SBC users are also inter-ested in providing additional media format translation inaddition to the stream management. Even simple require-ments like DMF tone monitoring require that the mediastreams are decoded and analyzed.

Te ability to have voice transcoding within the box helpssimplify the communications ow for an operator, henceprovides a competitive advantage for the equipment vendor.Unfortunately, voice and especially video stream processing

By Brian Carr, Emerson Network Power

Accelerate Server-basedMedia ProcessingA PCI Express media processing accelerator card using DSP technology offersbenefits over additional servers for high-density voice and video processing in

network applications.

Figure 1: Session border controllers are gateways between heteroge-neous networks



SPECIAL FEATURE

in real time at high channel counts is a strenuous task, soadding this function can impose a significant reduction onthe processing power available to the main service leadingto a reduction in capacity.

Possible SolutionsAdding media processing functionality to an application canbe done in a number of ways:

existing software

-ware-accelerated transcoding

In the SBC plus voice transcoding example above, using anexternal media gateway is perhaps the simplest to envisage.

Te border gateway terminates principal traffic streams, andredirects media to the external gateway for transcode via ex-ternal ports. Media can come back into the border gatewayfor egress filtering. Te disadvantage is that this is costly,uses rack space and extra power, takes up valuable physicalnetwork interfaces off the border gateway, and still requiresapplication development that controls and configures mediastream handling on a stream by stream basis.

aking the media server plus HD video example above, us-ing an external HD conferencing device will be complexto manage, will take up additional rack space and power,and could be high cost. Te service application would needto be able to manage both systems in parallel, potentially

increasing complexity, management overhead, and OPEXcosts. Upgrade paths to newer compression schemes suchas H.265 may be limited.

Te other two solutions allow for this function to be takeninside the box.

An internal software solution, for instance using commer-cially available host media processing software, necessarilymakes use of internal processing resources.

In the case of voice transcoding, this may be a greatsolution for a moderate number of simultaneouschannels, however it does not scale effectively. Atupwards of 1200 simultaneous channels of G.729encoding, the software solution approaches 50%utilization of a typical server, starving the originalapplication of processing resource. Effectively thismeans that additional servers would be requiredto offer higher densities of voice transcoding, and

the cost of the commercial software that is usuallycharged on a per-channel basis soon mounts up.

Although it is possible to add more servers to addressthis issue, accepting a reduction in capacity even foran improvement in functionality is often difficult tomanage from a product line perspective. It results in adowngrade of capacity within the same product offer-ing, so cannot really be viewed as adding functional-

ity. Matters get even worse when considering field upgradessince a customer must accept that a given installation wouldno longer be able to carry the same traffic.

The SolutionA more elegant solution to the problem is to use a plug-inmedia processing accelerator to offload both audio and vid-eo processing from the server host.

Tis keeps the function internal to the network elementAND avoids the loss of central processing resource thatwould otherwise be required to run a fully software solu-tion. Ideally this would be able to take account of new voiceand video compression schemes as they emerge.In this case, using a plug-in media processing accelerator of-fers a true upgrade path.

DSP Offload Card

It is now possible to deploy PCI Express media processingboards that offer high-performance voice and video transcod-ing based on digital signal processing (DSP) technology. Someboards even offer voice and video processing firmware opti-mized for their DSP array. Application developers can interactwith these boards via a simple object-oriented application pro-grammers interface (API). Te transcoding performance scaleslinearly according to the number of DSPs that are fittedop-tions from 4 DSPs to 12 DSPs are available. But even with 4DSPs and consuming less than 25W of power, cards are avail-able that deliver a voice transcoding performance comparableto a typical server consuming 300W or more.

Figure 2: A conventional host media processing (HMP) solution can compromiseserver performance

Figure 3: Emersons PCIE-8120 is an example of a PCI Express mediaprocessing accelerator card



SPECIAL FEATURE

An Example Application

An example may help illustrate the value of using accel-eration. Consider a packet processing application that, in aserver based on dual Intel Xeon processors, can support4000 concurrent sessions or streams. Te market now de-mands to add voice transcoding capability.

As outlined above, one option is to use a commercial host

media processing solution. Tis requires approximately50% of a dual Intel Xeon server capacity for 2000 transcodestreams. As a consequence, adding this capability reducesthe available processing power for the original applicationby 50%. Te resulting solution is now only a 2000 streamprocessing device. o get back to the 4000 stream capacity,a customer must buy two units, so power consumption andrack space is doubled.

Te alternative is to add a PCI Express media acceleratorcard. Tis takes care of the processing-intensive workload,thus maintaining the original performance. In fact, com-pared to a host media processing solution that is limited toapproximately 2000 sessions per server, a single PCI Ex-

press media accelerator card may be capable of transcodingover 7500 bidirectional voice streams or over 300 mobilevideo streams in hardware, and multiple boards can be fit-ted to a single server.

Voice Capability

When considering the PCI Express accelerator card route, de-sign engineers should ensure their shortlisted solutions sup-port the following 3GPP, IU-, IEF and other voice codecs:

Uncompressed telephony: G.711 -law/A-law withAppendices I and II

G.726, G.727

Wireless network: GSM EFR, AMR and AMR-Wideband; EVRCand EVRC-B

Internet voice: iLBC, SILK (Skype), Opus [roadmap]

In addition, each voice channel should support echo cancella-tion, announcements, conferencing, mixing, and a full rangeof tone detection and relay functions.

Video Capability

HD (or other) video streams can be redirected within an ap-pliance to a PCI Express accelerator card and transcoding andconferencing can happen without making any use of existingprocessing resource. For example, some PCI Express mediaaccelerator cards can handle up to six 4-party video con-ference bridges where each participant uses H.264 720p at30fps. Tere are also cards that can handle resizing to andfrom 1080p.

Design engineers should ensure the solution they choosesupports the most common video compression schemes used

in communications, such as H.263 (legacy) and MPEG-4 forCIF, and H.264 at resolutions up to 1080, and is easily up-gradeable as newer compression schemes emerge.

Many rack mount servers are available in fully NEBS compli-ant, hardened versions, so the accelerator card should be de-

signed for NEBS carrier grade and data center environments,so offering a common solution for both enterprise and tele-com environments.

A Better Solution

High density voice and video processing is increasingly indemand for applications such as session border controllers,media gateways/servers or media resource functions, videoor content optimization, video communications servers, andinteractive voice and video response systems. We can see thatusing a PCI Express media processing accelerator card ratherthan additional servers has a lot of benefits:

It consumes much less power

It can easily be retro-fitted to existing deployed systems asa true feature addition

It costs less than a comparable server + commercial hostmedia processing combination for the same performance

Consequently, it offers a lower total cost of ownership and a muchsimpler upgrade and deployment experience. In the words of Spi-nal aps Nigel, Tey go to eleven. Teyre one better.

Brian Carr is strategic marketing manager forthe Embedded Computing business of Emer-

son Network Power, with a particular focus oncommunications markets and applications in-cluding wireless, wireline and service delivery.

A widely published author and accomplishedspeaker on AdvancedCA technology and applications,Carr has also represented Emerson on conference advisoryboards and industry consortia. He holds masters degrees inengineering from Cambridge University and in informationtechnology from Essex University.



SPECIAL FEATURE

Te big news from the PCI-SIG is speed. From PCI to PCI Expressto Gen3 speeds, the PCI-SIG is an industry consortium thatlets no grass grow for long. As the embedded, enterprise andserver industries roll out PCIe Gen3 and 40G/100G Ethernet,the PCI-SIG and its key constituents like Cadence, Synopsis,LeCroy and others are readying for another speed doubling to

16 G/s (giga transfers/second) by 2015.

Te PCIe 4.0 next step would likely become known asGen4 and it evolves bandwidth to 16Gb/s or a whopping64 GB/s (big B) total lane bandwidth in x16 width. TePCIe 4.0 Rev 0.5 specification will be available Q1 2014with Rev 0.9 targeted for Q1 2015.

Yet as SIG-nificant as this Gen4 announcement is, PCI-SIGpresident Al Yanes said its only one of five major news items.

Five PCI-SIG announcements at DevelopersConference , June 2013

Te other announcements include: a PCIe 3.1 specificationthat consolidates a series of ECNs in the areas of power,performance and functionality; PCIe Outside the Boxwhich uses a 1 - 3 meter really cheap copper cable calledPCIe OCuLink with an 8G bit rate; plus two embedded andmobile announcements that Im particularly enthusedabout. See able 1 for a snapshot.

New M.2 SpecificationOne of two announcements for the mobile and embeddedspaces, the new M.2 specification is a small, embeddedform factor designed

to replace the previousMini PCI in MiniCard and Half MiniCard sizes (Figure 1).Te newer, as-yet-pub-licly-unreleased M.2card specification willdetail a board thatssmaller in size andvolume, but is intendedto provide scalablePCIe performance to

allow designers to tune SWaP and I/O requirements. PCI-SIG marketing workgroup chair Ramin Neshati told methat M.2 is part of the PCI-SIGs deliberate focus on mobilein a fundamentally changing market.

PCI-SIG-nificant ChangesBrewing in Mobile and SmallForm Factor DesignsOf five significant PCI Express announcements made at the PCI-SIG Develop-ers Conference, two are aimed at mobile embedded. Its about time.

By Chris A. Ciufo, Editor-in-Chief

Figure 1: The PCI-SIGs impending M.2 form factor is designed formobile embedded ultrabooks, tablets, and possibly smartphones.The card will have a scalable PCIe interface and is designed for Wi-Fi,Bluetooth, cellular, SSD and more. (Courtesy: PCI-SIG.)

PCI Express 3.1 specification Combines engineering change orders into new 3.1 spec for protocol exten-

sions, L1 power substates, lightweight notification, enhanced downstream

port containment, precision time measurement, more

PCI Express 4.0 specification Doubles bandwidth to 16 GT/s, 16 Gbps link, and about 64 GB/s total band-

width (x16).

PCIe Outside the Box Cheap, 1-3m copper cable starts at 8G bit rate with up to 32 Gbps each

direction (x4). Think of it as eSATA for PCIe and used for internal/external

storage. Will be orders of magnitude cheaper than Thunderbolt, says PCI

SIG spokesman

M.2 specification Replaces PCI Mini cards and designed for I/O modules in ultrabooks, tables

and possibly smartphones. Scalable PCIe I/F.

M-PCIe Mobile PCIe uses MIPI M-PHY in a smartphone to connect host ASSP to

modem, WLAN, and possibly onboard mass storage.



SPECIAL FEATURE

Mobile PCIe (M-PCIe)Te momentum in mobile and interest in a PCIe on-boardinterconnect lead the PCI-SIG to work with the MIPIAlliance and create Mobile PCI Express: M-PCIe. Te speci-fication is now available to PCI-SIG members and createsan adapted PCIe architecture bridge between regular

PCIe and MIPI M-PHY (Figure 2).

Using the MIPI M-PHY physical layer allows smartphoneand mobile designers to stick with one consistent userinterface across multiple platforms, including already-existing OS drivers. PCIe support is baked into Windows,iOS, Android and others, says PCI-SIGs Neshati. PCIExpress also has a major advantage when it comes tointeroperability testing, which runs from the protocolstack all the way down to the electrical interfaces. akencollectively, PCIe brings huge functionality and compli-ance benefits to the mobile space.

M-PCIe supports MIPIs Gear 1 (1.25-1.45 Gbps), Gear 2(2.5-2.9 Gbps) and Gear 3 (5.0-5.8 Gbps) speeds. As well, theM-PCIe spec provides power optimization for short channelmobile platforms, primarily aimed at WWAN front end radios,modem IP blocks, and possibly replacing MIPIs own universalfile storage UFS mass storage interface (administered by

JEDEC) as depicted in Figure 3.

PCI Express Ready for MoreMore information on these fiveannouncements will be rolling out soon.But its clear that the PCI-SIG sees mobile

and embedded as the next target areas forPCI Express in the post-PC era. Yet theorganization is wisely not abandoningthe PCI Express standards bread andbutter in high-end/high-performanceservers and systems.

Chris A. Ciufo is editor-in-chief for embedded contentat Extension Media, whichincludes the EECatalog print

and digital publications andwebsite, Embedded IntelSolutions, and other related blogs and embed-ded channels. He has 29 years of embeddedtechnology experience, and has degrees inelectrical engineering, and in materials sci-ence, emphasizing solid state physics. He canbe reached at cciufo@extensionmedia.com.

Te scalable M.2 card is designed as an I/O plug in for Bluetooth,Wi-Fi, WAN/cellular, SSD and other connectivity in platformsincluding ultrabook, tablet, and maybe even smartphone, saidNeshati. At Rev 0.7 now, the Rev 0.9 spec will be released soon andthe final (Rev 1.0?) spec will become public by Q4 2013.

Figure 2: The Mobile PCI Express (M-PCIe) specification targetsmobile embedded devices like smartphones to provide high-speed,on-board PCIe connectivity. (Courtesy: PCI-SIG.)

Figure 3: M-PCIe by the PCI-SIG can be used in multiple high speed paths in a smartphone mobiledevice. (Courtesy: PCI-SIG and MIPI Alliance.)



SPECIAL FEATURE

odays mil/aero systems are decidedly in the world ofmulticore processing. Mainstream processors are availablewith 10 and 12 cores. Slightly more exotic architecturesare available with tens to hundreds of cores (Xeon Phi,ile, GPGPU). Tis allows abundant processing power to

be applied to problems, but how to keep the cores fed withdata? An idle core is a wasted resource. Its not just band-width between resources that is of concernits also thelatency incurred in transfers. Tat can significantly affectthe bandwidth for small transfers and can also make thesystem-transfer function exceed its design requirementsfor real-time operation. Newer architectures allow fortechniques such as remote direct memory access (RDMA)and methods that build upon it such as GPUDirect to beemployed to address these issues and to build systems thatbalance data movement with processing power.

Te underlying premise of high-performance embedded

computing (HPEC) is to leverage the architectures andsoftware that are prevalent in the world of commercialsupercomputing and to apply them to the harsh world ofmil/aero applications. Tis has led to a widespread adop-tion of processors from Intel, AMD and NVIDIA andstandard interconnects such as PCI Express, Ethernet andInfiniBand. However, vanilla implementations can fallshort of expectations for performance in these demandingapplications. For instance, running a CP/IP stack for a10GbE link can easily burn up an entire processor core withno cycles left over for computation workloads. Similarly,the measured performance can fall far short of theoretical

wire-speed maximums.

Sensor to ProcessorOn the sensor side, it is common to see FPGAs rightbehind the analog-to-digital converters, with some kindof data link to the processing system. While it is possibleto program FPGAs to talk to most of the major protocols,it is most common to see either PCI Express (which ishard-coded into many devices) or a simple interface suchas serial FPDP. In systems where the analog acquisition isco-located with the processing, PCI Express is a common

choice, but can suffer from excessive latency and multipletransfers unless some key techniques are employed.

For example, a couple of years ago it was not uncommon tosee the data undergo three transfers to get from the sensorto GPU memory ready for processing. Tis translates intolarger latency than may be desired, and to reduced overallbandwidth due to the data being moved three times. Tis

Multiple Processors inHigh-Performance EmbeddedComputing: Only ConnectMultiprocessing is at the heart of high-performance embedded computingin mil/aero systemsand new technologies enable it to deliver significantly

improved performance.

By Peter Thompson, GE Intelligent Platforms

Figure 1: Comparison of measured bandwidth between 10GbE usingstandard TCP and RoCE RDMA

Figure 2: Comparison of measured latency between 10GbE using stan-dard TCP and RoCE RDMA



SPECIAL FEATURE

may be tolerable for some applications (such as syntheticaperture radar, where the acquisition of a complete dataset takes a long time, and the calculation of an image isnot generally needed immediately), but not for others suchas tactical radars and electronic warfare systems where a

signal must be received, processed and sent out in as littletime as possible.

Te solution to thisproblem lies in technologytermed GPUDirect RDMA.Te first iteration of GPU-Direct removed the hostmemory-to-host memorytransfer by allowing acommon address space,eliminating one wholetransfer. Tis reduced

latency by around onethird. With the introduc-tion of the Kepler familyof GPUs and the latestversion of CUDA (5.x),NVIDIA removed anothertransfer step, and nowallows the PCI Express endpoint and the GPU to see a pieceof each others memory, allowing either device to transferdirectly to or from the other. Tis not only removes theneed for an extraneous transfer, but also removes the hostprocessor and operating system from the chain. Measure-

ments have shown this to reduce latency by a factor of 15,opening up new application areas for GPGPU. It is worthnoting that none of this requires proprietary mechanisms.Te interface is standard PCI Express; the transfers useregular DMA mechanisms. It just happens to be that thedestination address is an aperture into GPU memory thathas been mapped to PCI Express space and is exposed viaa simple API call.

Processor to ProcessorWhen it comes to communicating between processors ina multiprocessor system, there are many choices. Intel

devices may be connected with QuickPath Interconnectin a symmetrical multi-processing architecture thatmakes two processors appear to the operating system asa single large CPU. However, this functionality is gener-ally not available in the devices that are provided in theBGA packages from the extended lifecycle portfolios thatare favored for rugged environmentsonly in socketedversions that are not fully rugged or must go through a

complex rework process.

Between boards, most of the major fabric choices areavailable, but the right choice can have a marked effecton system performance. For example, if standard CPsockets over 10GbE are used, substantial CPU resourcesare used up just to manage the transfersand the datarates achieved are less than optimal. If, however, remotedirect memory access (RDMA) is available, the picturechanges dramatically. RDMA allows the network interfacechips to move data directly from the application space onone board to that on another with no extra copies and

no involvement of the host processor once the transferhas been requested. Tisreduces the amount ofCPU cycles used to managethe transfer to single digitpercentages rather thanclose to one hundred per-cent. Examples are RDMAover Converged Ethernet(RoCE) from Mellanoxand Internet Wide AreaRDMA Protocol (iWARP)from Intel. Both have

low latency (as low as 1.3microseconds for RoCE, 3microseconds for iWARP).Mellanox also supportsRDMA over InfiniBand(using the same networkinterface controllers

(NICs)), allowing the system designer to choose betweenthe two fabrics. Te API to the application remains thesame thanks to the OFED (Open Fabrics Enterprise Distri-bution: open source software for RDMA and kernel by passapplications) stack. (Figure 1, Figure 2).

GPU to GPUSimilar data movement issues arise in a heterogeneousHPEC system comprised of one or more host processorsand multiple GPUs. In such a system, it is common to viewthe GPUs as the primary data consumers, with the hostprocessor(s) being relegated to doing housekeeping func-tions. Once the data has been deposited into a GPU from asensor interface (using GPUDirect RDMA), after a first stageof processing it may be necessary to move it to another GPUfor the next stage of a pipelined processing scheme.

Figure 3: The GE Intelligent Platforms IPN251 is a 6U OpenVPX mul-tiprocessor that combines an Intel Core i7 processor with a 384-coreKepler GPU from NVIDIA

The underlying premise of

high-performance embedded

computing (HPEC) is to

leverage the architectures and

software that are prevalent in

the world of commercial su-

percomputing and to applythem to the harsh world of

mil/aero applications.



SPECIAL FEATURE

In the past, this would require the data to be movedfrom GPU memory to its host system memory, then per-haps to another host processor, then finally down to the

memory of the destination GPU. Again, multiple trans-fers, extra latency, wasted bandwidth and unwantedburden on the hosts. And again, GPUDirect RDMAcomes to the rescue via two forms of support. Firstly,if the source and destination GPUs reside in the samePCI Express domain, an RDMA transfer is availablefrom GPU memory to GPU memory across PCI Express.his can occur whether the GPUs are both connecteddirectly to the same host root complex, or if they areboth downstream of a PCI Express switchalthoughthe latter scenario can lead to better performance.

What if the GPUs are not is the same PCI Express domain?

For example, consider two GE Intelligent PlatformsIPN251s in a system. (Figure 3, Figure 4). Each IPN251has a 3rd Generation Intel i7 CPU connected to a 384-coreKepler GPU via 16 lanes of Gen3 PCI Express. Each CPU/GPU cluster also has a Mellanox ConnectX-3 NIC pro-viding two channels of 10GbE and/or DDR InfiniBand forinter-board communication. By leveraging another formof GPUDirect RDMA, it is possible now for one GPU toRDMA data to a remote GPU across PCI Express (from GPUto NIC) and InfiniBand (from NIC to NICmost likely viaa system level switch). Again, this avoids multiple trans-fers and does not require host intervention.

CPU to GPUTese transfers have always allowed for DMA across thePCI Express bus between system and GPU memory, gener-ally relying on a DMA engine built into the GPU.

The complete systemBy combining these mechanisms, it is now possible toconstruct an HPEC system for processing sensor data for avariety of mil/aero applications including radar, electronicwarfare, synthetic vision and many others that not onlyoptimizes processing power by mixing FPGAs, general pur-

pose processors and GPUs, but also optimizes system dataow to minimize latency, maximize bandwidth utilization,and maximize processor utilization for computation rather

than data movement.

Open architecture softwareWhile the low-level implementations of these transfermechanisms may use proprietary APIs, in many casesit is possible to use them via high level open-standardAPIs. Now, portable OSA applications can leveragethe performance boosts of RDMA without having anyhardware-specific ties. For example, the message passinginterface (MPI) library can be layered over OFED, allowingapplications using this commonly used API to take advan-tage of RDMA transfers between processors without anyhardware-specific code. Similarly, MPI transfer primitives

can copy data directly between CPU and GPU memory. Anoptimized MPI library can differentiate between devicememory and host memory without any hints from theprogrammer and can use the appropriate RMDA transfers.

Te different RDMA methods described (sensor to GPU, CPUto CPU, GPU to GPU, CPU to GPU) are frequently employedat the same time in a system to optimize end-to-end datamovement. Te result can be a huge increase in performanceof the system, allowing reductions in SWaP, increased func-tionality, or enabling applications that were previously not agood fit for such programmable, scalable architectures.

Peter Tompson is senior business develop-ment manager, High Performance EmbeddedComputing at GE Intelligent Platforms. Withan honors degree in electrical and electronicengineering from the UKs University of Bir-mingham, Peter has worked for over 30 yearsin embedded computing. defense.ge-ip.com.rillium, andObjectStream. Drew has a BA in human services fromWestern Washington University in Bellingham, WA.

Figure 4: Two IPN251s process data received from a GE ICS-1572ADC/DAC transceiver module with data being passed from one GPUto another via RDMA.



SPECIAL FEATURE

USB 3.0 is ubiquitous in new PCs as astandard feature, but its commonlyconsidered to have come later to themarket than consumers would havepreferred. Anyone who purchased anew Apple iPod and had to endure the

transfer 10+ GB of data over a USB 2.0connection would have preferred USB3.0 speeds. Digital cameras take andstore pictures that are 3 MB per imageand more. Digital video cameras recordvideo at the rate of about 10 minutesper gigabyte. HD video recorders use araw file format that records even moredetail in an uncompressed format,which creates noticeably larger files. Sowhile USB 3.0 is ubiquitous and popular in new PCs, it wouldhave been just as needed and popular three years ago. Whilewe could ponder on the question, Why is USB 3.0 so late to

the market? its more informative for us to consider whatconsumers will need in 3 years.

Trends in Storage and PricesTe capacity of hard disk drives (HDDs) and solid state drives(SSDs) continues to increase, and prices per gigabyte todecrease. While HDDs will be the common way to store hugeamounts of data, the price of SSDs will drop more quickly.Te prevalence of instant on smartphones and tablets,which take advantage of SSD performance, has made con-sumers want the same from their laptops. SSDs also have theadvantage of consuming less power which extends battery

life. Basically, consumers are used to the performance andbattery life offered by their devices using SSD, and this willdrive even greater demand for SSDs over HDDs.

In addition to instant on performance and low powerconsumption, the price of SSDs is consistently going down,even with increasing performance. oday, a 256 GB SSDfrom Crucial or Sandisk sells for about USD$170 retail.Tis SSD can read data at 5.4 Gbps and write data at 4.4Gbps using SAA 6 Gbps interfaces, which are faster thanthe effective USB 3.0 throughput rates of 3.2 to 3.5 Gbps.Assuming prices drop by about 50 percent each year (as

has been the trend so far), the retail price of a 256 GB SSDwill drop to $21 within 3 years (Figure 1). We can assumea retail margin of 30 percent for the retailer (not the ash

memory maker), so the estimated cost of this memory tothe SSD maker is $120.

And so, if the cost to the manufacturer is about $120 today, thecost for that same memory will drop 50 percent a year to about$15 by 2016. Even if the drop is only 30 percent a year, the costdrops to about $29. Either way, the integration of this muchmemory is highly compelling to consumer device manufac-turers of cameras, smartphones, tablets, and phablets (i.e.,smartphones with almost tablet-sized screens).

Using a similar extrapolation which assumes the doubling

of memory capacities each year at the same price point,$170 will buy a 2 B SSD in 2016 (Figure 2).

Looking at this trend, tablet and smartphone designersare making the obvious choice to increase their productsstorage capacity. More importantly, system architects in awide variety of consumer industries are integrating SSDsinto their products while increasing their products appeal byintegrating higher-quality digital video cameras and digitalcameras. Larger files, from photos and videos, in greatercapacities of high-performance memory, leads to consumer

Figure 1: Retail price and cost of 256 GB SSD with 50% and 30% price drops per year

The Accelerating Demand for

10 Gbps SuperSpeed USB 3.0

By Eric Huang, Synopsys



SPECIAL FEATURE

demand to move the data more quickly. So begins the battleof the standards.

Competing StandardsIn 2012, Apple launched monitors and PCs with the Tun-derbolt standard, which supports 10 Gbps transfer speeds.As usual, Apple is ahead of everyone else. Tunderbolt sup-ports video and data multiplexed through a single cable.MacBook Air or Pro PCs can be connected to either an Applemonitor or a Tunderbolt docking station. Te same Tun-derbolt cable carries a multiplexed 10 Gbps PCI Express(PCIe) connection that can carry data from the MacBook

to the monitor and other devices plugged into the monitoror docking station. In addition, the PCIe connection allowsthe docking station or monitor to integrate a USB 3.0 hostto connect to USB peripherals and/or Tunderbolt deviceslike HDDs. In this case, the USB 3.0 host is in the dockingstation, not inside the PC.

Apple uses Tunderbolt to fill its customers need for a fastexternal interface. In fact, the next generation of Tunderboltwill go to 20 Gbps to carry 4K video to meet the demand fora faster interface. Te problem is that Tunderbolt is a closedstandard. It requires discrete chips in both the host and client.

It requires an active cable with chips embedded in the cable.Adding Tunderbolt increases a systems cost by $10-25.

On the other hand, integrating USB 3.0 has a much lower pricepoint. In fact, every major PC manufacturer currently pro-duces a USB 3.0 docking station based on a DisplayLink chip.Te docking stations allow consumers to use existing USB 3.0drivers concurrently with existing USB 2.0 and 3.0 peripherals.One USB 3.0 cable connected to a USB 3.0 docking station cansupport multiple USB 1.1, 2.0, and 3.0 peripherals, in additionto HDMI monitors. ASUS, HP, Fujitsu, oshiba, Lenovo, and

Engineers Guide to PCI Express Solutions

Documents