MHz Digital Frequency Domain MHz Digital Frequency Domain Multiplexer Multiplexer Matt DobbsMatt Dobbs
McGill UniversityMontreal, Canada
Berkeley (LBNL/UC) (original Analog fMux, cold components for all systems)• John Clarke, Bill Holzapfel, Adrian Lee, Paul Richards, Helmuth Spieler• Sherry Cho ( NIST)• Trevor Lanting ( McGill), Martin Lueker, Tom Plagge• John Joseph, Chinh VuMcGill (new Digital Backend, high BW SQUIDs)• Francois Aubin, Peter Hyland, Kevin MacDermid• Eric Bissonnette, Graeme SmecherMinnesota (cooling, striplines, bolo integration) NIST (SA SQUIDs, Inductors)• Shaul Hanany, Hannes Hubmayr ● Gene Hilton, Kent Irwin
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 2
fMuxfMux in the Fieldin the Field
EBEX1500 TES, 2009Digital fMux
SPT1000 TES, 2007
Analog fMux
POLARBEAR1200 TES, 2009
Digial fMux
APEX320 TES, 2006
Analog fMux
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 3
fMUX Readout SystemfMUX Readout SystemStaged Development and DeploymentStaged Development and Deployment
APEX-SZ320 Channels2005, Chile
POLARBEAR1200 Channels
South Pole Telescope1000 Channels
2006/7EBEX
1500 ChannelsLD Balloon
AC-bias, Scalable SystemMUX Demonstration
Full MultiplexerLow Power, Flight Worthy2nd Generation
Digital
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 4
SQUID
250 mK
TES
L
G
Optical Power
Basic TES ReadoutBasic TES Readout
Vbias
Warm Bi-polaramplifier
Vout
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 5
SQUID
250 mK
TES
L
G
Optical Power
Basic TES ReadoutBasic TES Readout
Vbias
Warm Bi-polaramplifier
Vout
AC
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 6
MHz Frequency Domain MultiplexerMHz Frequency Domain Multiplexer
f500 kHz
f500 kHz
200 Hz
80 dBc
Amp-Modulated by BoloDDSAmplified
600 kHz 600 kHz f100 Hz
Demodulated
500 kHz
600 kHz
f100 Hz
f500 kHz
200 Hz
80 dBc
600 kHz
Shunt FBSQUID
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 7
MHz Frequency Domain MultiplexerMHz Frequency Domain MultiplexerAdvantage #1: Simple architecture.
Continuous readout, no switching transients.Only 1 stage of SQUIDs. 1 SA-Squid per module.No SQUIDs in focal plane – straight forward magnetic shielding.
Easy SQUID setup/tuning, easy to understand.
Wire count:Two sub-Kelvin wiresper module.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 9
fMuxfMux –– Cryostat ComponentsCryostat Components
MUX Inductors
TES WedgeSQUID board
Cryoperm SQUID Shield
4K room Wiring
SQUID Controller(room temp)
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 10
Cold Component AdvantagesCold Component AdvantagesNO power dissipated on subNO power dissipated on sub--Kelvin stageKelvin stage..
Readout system components are LReadout system components are L’’s and Cs and C’’s only.s only.Optimized for Optimized for RRnormalnormal=0.5=0.5--1 1 ΩΩ
Small impedance of connectors between bolos & SQUIDs not an issue.
No bump bonds or complexity in the interface.Individual bias levels for each bolometerIndividual bias levels for each bolometer
Shifts system complexity away from bolo Shifts system complexity away from bolo fabfab..Sky signals are modulated at MHzSky signals are modulated at MHz
Well above Well above microphonicsmicrophonicsNo direct readout sensitivity at baseband, where SQUID is sensitNo direct readout sensitivity at baseband, where SQUID is sensitive to ive to magnetic fieldsmagnetic fields
Only Only one SQUID stageone SQUID stageLives at 4K, easy to shield.Lives at 4K, easy to shield.No SQUIDs on focal plane.No SQUIDs on focal plane.Easy to characterize. No multiple tunings.Easy to characterize. No multiple tunings.Gain is set by warm FLL feedback resistor Gain is set by warm FLL feedback resistor –– not necessary to characterize RO not necessary to characterize RO gain channel by channel.gain channel by channel.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 11
11stst Generation Analog MUXGeneration Analog MUX
SQUID
2 modules of 8 channelseach
4 modules of 16 channelseach
New Digital MUXNew Digital MUXAdvantage: Complete modularity.
1 SA-Squid = 1 Module.No columns/rows or cross bars.
If one component fails, you loose one module only.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 12
Channel Spacing/CountChannel Spacing/Count
LLMUXMUX dictated by bolometer time dictated by bolometer time constants.constants.
If If ττ TESTES << << ττopticaloptical, channels will , channels will need to be widely spaced.need to be widely spaced.Presently we use LPresently we use LMUXMUX = 16 = 16 μμHHThis can be reduced by factor 3This can be reduced by factor 3--5 5 by careful tuning of bolometer by careful tuning of bolometer time constants.time constants.
ΔΔff chosen to ensure thatchosen to ensure thatJohnson noise leakage from Johnson noise leakage from neighboring channels is small neighboring channels is small (presently 2%)(presently 2%)Neighbor cross talk is small Neighbor cross talk is small (presently < 10(presently < 10--33))Bias leakage from neighbors is Bias leakage from neighbors is <10% (spoils voltage bias)<10% (spoils voltage bias)
With 16 With 16 μμHH inductors, this inductors, this spacing is 75 kHz today.spacing is 75 kHz today.
Channel count is just BW/spacingChannel count is just BW/spacing
BW is limited by SQUID FLL, BW is limited by SQUID FLL, which presently uses a 4K SQUID which presently uses a 4K SQUID
300K amplifier feedback.300K amplifier feedback.
Room for substantial Room for substantial improvementimprovement
Optimize bolo Optimize bolo ττCould envision handling bias Could envision handling bias leakage with active feedbackleakage with active feedbackGreatly increase BW with Greatly increase BW with alternate SQUID FLL (factor 10 alternate SQUID FLL (factor 10 has already been achieve in the has already been achieve in the lab lab ““LISALISA””))
Advantage #3: System is “young”, with established directions for improvement on short timescale if adequately funded.
No fundamental limit to the number of MUX-ed channels per comb.(one might argue for at least a factor 10 in modularity – target 100?)
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 13
Electronics AnatomyElectronics Anatomy
Two MezzaninesPer MotherboardProvide 4 MUXCombs.
Advantage (AGAIN): Complete modularity.All control + smarts implemented onembedded processor.
His board needs NOTHING from the outside world except a clock and power to tune its module and keep it going. There is no special master board.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 14
System SetupSystem SetupNetwork AnalysisNetwork Analysis
System measures the System measures the complex impedance of the complex impedance of the MUX combMUX comb
Accounts for changes in C as Accounts for changes in C as system coolssystem cools
Chooses bias points.Chooses bias points.
Takes about 15 minTakes about 15 minNo attempt has been made to No attempt has been made to make this fast.make this fast.
This algorithm needs to be This algorithm needs to be run ONCE in the lifetime of run ONCE in the lifetime of each comb.each comb.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 15
System SetupSystem SetupSQUID TuningSQUID Tuning
Only one SAOnly one SA--SQUID to tune SQUID to tune per combper comb
Trace out VTrace out V--phi curve as a phi curve as a function of function of IIbiasbias..
Choose optimal bias point Choose optimal bias point and lock FLL.and lock FLL.
Long version (once per Long version (once per SQUID lifetime) takes 2 SQUID lifetime) takes 2 minutes.minutes.
Short version (once per Short version (once per SQUID thermal cycle) takes SQUID thermal cycle) takes ½½--1 min. Further 1 min. Further improvements possible.improvements possible.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 16
System SetupSystem SetupBolo TuningBolo Tuning
Heat or zap bolos normal.Heat or zap bolos normal.
OverOver--bias bolometers, and bias bolometers, and null carriersnull carriers
Drop bolos into transition, Drop bolos into transition, mapping out Imapping out I--V curveV curve
Can also map out complex Can also map out complex impedance impedance Z(Z(ωω), to measure ), to measure response function and bolo response function and bolo time constants.time constants.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 18
System Performance System Performance -- NoiseNoiseComponent of white noise from Component of white noise from Readout system is enhanced by Readout system is enhanced by factor factor √√22
true for any AC biased systemtrue for any AC biased systemIn a well optimized system the In a well optimized system the contribution of this noise should contribution of this noise should be negligible.be negligible.
Digital system has significant Digital system has significant improvement in 1/f noiseimprovement in 1/f noise
Knee <= 100 mHz, dominated by Knee <= 100 mHz, dominated by DAC output DAC output transisitorstransisitors..This noise is COMMON mode This noise is COMMON mode across a comb.across a comb.Can monitor it with dark channel Can monitor it with dark channel or subtract with the typical or subtract with the typical common mode subtraction used common mode subtraction used for temperature fluctuations and for temperature fluctuations and atmosphere.atmosphere.
SPT 2008, Measured noise on sky.(E. Shirokof, ASC 2008 talk)
fMuxfMux noise demonstrated noise demonstrated with SPT.with SPT.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 19
System Performance System Performance -- NoiseNoiseComponent of white noise from Component of white noise from Readout system is enhanced by Readout system is enhanced by factor factor √√22
true for any AC biased systemtrue for any AC biased systemIn a well optimized system the In a well optimized system the contribution of this noise should contribution of this noise should be negligible.be negligible.
Digital system has significant Digital system has significant improvement in 1/f noiseimprovement in 1/f noise
Knee <= 100 mHz, dominated by Knee <= 100 mHz, dominated by DAC output DAC output transisitorstransisitors..This noise is COMMON mode This noise is COMMON mode across a comb.across a comb.Can monitor it with dark channel Can monitor it with dark channel or subtract with the typical or subtract with the typical common mode subtraction used common mode subtraction used for temperature fluctuations and for temperature fluctuations and atmosphere.atmosphere.
Noise spectra for EBEX bolometers(H. Hubmayr, see poster)
fMuxfMux noise demonstrated noise demonstrated with SPT with SPT Lab demonstrations of Lab demonstrations of Digital system.Digital system.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 20
DfMuxDfMux Size/Form FactorSize/Form FactorOne 6U VME crate houses 20 One 6U VME crate houses 20 DfMuxDfMuxboards (80 modules)boards (80 modules)
16x MUX = 1280 channels16x MUX = 1280 channels
No wire length restrictionsNo wire length restrictions
Only connection to outside is via Ethernet.Only connection to outside is via Ethernet.
Interfaces to 10 SQUID Control boards Interfaces to 10 SQUID Control boards mounted on side of cryostat in small (7mounted on side of cryostat in small (7””cube) RF enclosurescube) RF enclosures
Present design requires short wire separation Present design requires short wire separation from SQUIDs, 20cm.from SQUIDs, 20cm.
Open loop SQUID operation (with active Open loop SQUID operation (with active nulling), or nulling), or LinearizedLinearized SQUID arrays (LISA) SQUID arrays (LISA) would remove this wire length requirement.would remove this wire length requirement.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 21
Tech Challenge: Wiring InductanceTech Challenge: Wiring InductanceFor passive voltage bias, RFor passive voltage bias, RBOLOBOLO
must dominate.must dominate.
Wiring Wiring LLstraystray cannot be tuned cannot be tuned away.away.
Analog systems used short lengths Analog systems used short lengths of of NbNb/Ti twisted pair (/Ti twisted pair (bad,badbad,bad).).
New system uses New system uses NbNb striplinesstriplinesdeveloped at Minnesota.developed at Minnesota.
Keeps Keeps LLstraystray negligible for negligible for RRbolobolo ≈≈ ½½ ΩΩ
Allows for long wires from Allows for long wires from subKelvinsubKelvinto 4K.to 4K.
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 22
Tech Challenge: Wire LengthsTech Challenge: Wire LengthsPresently SQUID FLL includes Presently SQUID FLL includes components at 300K and 4Kcomponents at 300K and 4Kphase delay along these wires restricts phase delay along these wires restricts wiring to 20cm (ouch wiring to 20cm (ouch –– big heat load).big heat load).Real issue: preReal issue: pre--amplifier bandwidthamplifier bandwidth
(my favorite) Strategies:(my favorite) Strategies:Move some (or all) of the feedback to 4KMove some (or all) of the feedback to 4K
SQUID OpSQUID Op--ampampLinearizedLinearized SQUID Array SQUID Array
Run the squids open loop Run the squids open loop –– and keep the bias and keep the bias carriers tiny.carriers tiny.
Digital feedback to control nulling amplitudeDigital feedback to control nulling amplitude
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 23
Tech Challenge: Power ConsumptionTech Challenge: Power Consumption
Power scales with number of modules(!)Power scales with number of modules(!)DSP power per Watt for DSP power per Watt for FPGAsFPGAs is increasing very fast.is increasing very fast.
Substantial power savings could be achieved by:Substantial power savings could be achieved by:Reducing data conversion rate from 25 MHz to 5 MHz.Reducing data conversion rate from 25 MHz to 5 MHz.Increasing number of channels/Modules. Increasing number of channels/Modules.
32 32 chanchan/module = 130 W//module = 130 W/kilopixelkilopixel128 128 chanchan/module = 33 W//module = 33 W/kilopixelkilopixel
Achieving this requires careful control of bolo time constants aAchieving this requires careful control of bolo time constants and/or a nd/or a larger scale engineering effort on par with the UCB TDM effort.larger scale engineering effort on par with the UCB TDM effort.
Predicted Measured Per Mod [W]# MUX Mods [W] [W] [W]
DfMUX FPGA Motherboard 4 5.8 6.1 1.5DfMUX Analog Mezzanine 2 4.0 4.3 2.2MUX SQUID Controller 8 4.1 4.0 0.5
total per MUX module 4.175
total for 32 modules (384 channels assuming 12x MUX) 134Power consumption per kilopixel 348
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 24
Space QualificationSpace QualificationSQUIDs:SQUIDs:
Flight qualified for Gravity Probe B.Flight qualified for Gravity Probe B.
Need radiation hardness test of NIST SANeed radiation hardness test of NIST SA--SquidsSquids
FPGAsFPGAs::Xilinx V4 family already has flight qualified line, V4QV. IncludXilinx V4 family already has flight qualified line, V4QV. Includes es device 20% larger than that used at McGill (LX200).device 20% larger than that used at McGill (LX200).
McGill/COM DEV funded through CSA to develop firmware SEL/SEU McGill/COM DEV funded through CSA to develop firmware SEL/SEU error detection and correction technology.error detection and correction technology.
A/D and D/A are biggest challengeA/D and D/A are biggest challenge25 MHz devices used in McGill 25 MHz devices used in McGill DfMuxDfMux system not flight qualified.system not flight qualified.
In principle, no features exclude this possibility In principle, no features exclude this possibility –– perhaps even for the perhaps even for the devices we use.devices we use.
our requirements are different from most commercial/communicatioour requirements are different from most commercial/communications ns markets.markets.
Technical readiness (via Technical readiness (via BenfordBenford method) method) ≡≡ 4.84.8
[email protected]@McGill.ca, , CMBpolCMBpol Aug 2008Aug 2008 25
SummarySummaryAdvantagesAdvantages
Simple. True modularity.Simple. True modularity.No subNo sub--Kelvin power dissipationKelvin power dissipation
Set bolo biases individuallySet bolo biases individuallySkySky--signals modulated above signals modulated above microphonicsmicrophonics and magnetic and magnetic pickuppickupOnly one SQUID stage (4K)Only one SQUID stage (4K)No formal limit on # MUX No formal limit on # MUX channels/module.channels/module.
Primary challenge is FLL Primary challenge is FLL bandwidth/wire lengthbandwidth/wire length
ChallengesChallenges
Power consumptionPower consumption1/f D/A noise at 100mHz1/f D/A noise at 100mHz
But purely common mode across But purely common mode across comb.comb.
11stst Generation Analog MUX in Generation Analog MUX in field for SPT/APEXfield for SPT/APEX
achieving excellent noise achieving excellent noise performance.performance.
New warm electronics deploying New warm electronics deploying for EBEX/for EBEX/PolarbearPolarbear
Relatively young.Relatively young.Scales with Scales with numbrnumbr of of MuxMuxModulesModulesImproved 1/f noise (<0.1Hz)Improved 1/f noise (<0.1Hz)10x less power, 10x smaller10x less power, 10x smaller