RF Machine Learning Systems (RFMLS) - DARPA · PDF fileRF Machine Learning Systems (RFMLS) ......

RF Machine Learning Systems (RFMLS)

Paul Tilghman

Industry Day

August 31st, 2017

Approved for Public Release, Distribution Unlimited

2

RFMLS Industry Day Agenda

9:00-9:15 Agenda and Logistics9:15-9:35 Vision9:35-9:55 RF Machine Learning Challenges9:55-10:50 Program Structure and Technical Areas10:50-10:55 Program Execution10:55-11:15 Break11:15-11:40 Proposal Evaluation Criteria and Proposal Guidance11:40-12:00 Contracting12:00-1:00 Lunch Break1:00-4:00 PM Meetings / Teaming Time4:00-5:00 Question & Answer Session

August 31, 2017


3

• Questions must be submitted in writing by 12:00 noon• Questions will be answered at 4:00 PM

• Answers to questions will be made available as a new attachment to the BAA• Questions may be submitted at a later date to [email protected]

Question & Answer Session


mailto:[email protected]

4

Schedule of Meetings with Program Manager

Future meetings or teleconferences with the PM may be requested through email to [email protected]

1:00-1:151:15-1:301:30-1:451:45-2:002:00-2:152:15-2:302:30-2:452:45-3:003:00-3:153:15-3:303:30-3:453:45-4:00


mailto:[email protected]

5

RFMLS Vision

Three Waves of Artificial Intelligence

1st WaveHandcrafted Knowledge Machine Learning

2nd Wave 3rd Wave:Contextual Adaptation

Data

Models

Analysis

Experts

Rules

Advice

Data

Models

Analysis

6

Rich collaboration between humans and machines enabled

by shared perceptions of the real world

Humans program systems with explicit rules or logic in

limited domains

Systems learn statistical models of specific problems

using big data


7

Existing Cognitive & Adaptive RF Systems

Digital Signal Processing AppRF Frontend

Data Reduction

1st Gen Cognitive EW (Today)Adaptive RF SystemsFirst generation confined due to hand-engineered

choices in DSP and frontendAdaptive RF system – without intelligence

First-generation cognitive/adaptive RF systems limited to 1st wave-AI

AI Wave 1Expert System

AI Wave 2Statistical

AI Wave 3Contextual

1st Gen Cognitive EWAdaptable RF Systems

Deep LearningExplanatory, Continual,Contextual, Important, …


8

RFMLS Establishes Foundation for 2nd Gen. Cognitive RF

App

RFMLS bridges the gap between modern data-driven machine learning and the spectrum

Wave 1Expert System

Wave 2Statistical

Wave 3Contextual

1st Gen Cognitive EWAdaptive RF Frontends

RFMLS

Digital RF Machine LearningReconfigurable RF FrontendLearning Feedback

Learning Feedback

RF dataset


9

Discrimination in the RF Spectrum

1 10 100 1000 10000 100000

Known-parametersIn-catalog

Known-parametersout-of-catalog

Learned-parametersout-of-catalog

Objects of importance

Context dependent recognition

Interactive recognition

Discriminability (population size)

Disc

rimin

atio

n Di

fficu

lty

Low error, 100 languages speech, deep nets (2017)

Large Vocabulary Continuous Speech neural nets (2000s)

ten isolated digits (1952)

MNIST handwritten digits (1998)

3D Blocks recognition

(1963)

AlexNet(Imagenet)

(2012)

DeepFace(Labeled faces

in the wild) (2014)

Show and Tell (2016)

Two-level Attention (2015)

DeepMind Atari, Data Centers (2015, 2016)

Cognitive RF Sensing

Hand-engineered fixed-systems

Traditional RF Sensing

Speech Recognition

Image Recognition

Handling signal diversity

Handling the data deluge from GHz

RF devices

Tuning the RF device to the environment


10

Inspiration for a Better RF/digital Frontend

Low-level Feature Rep

Sparse Encoding

Abstract Feature Rep

Pattern Rec & Memory

Cognitive Task

Bottoms-up saliency and data rate reduction Top-down attention and goal-driven feedback

SpeechHead turning

Eye Movement

Instinctive and overt motor control

~100 Gbps ~10 MbpsInfinite Information ~100 kbps Key characteristics for RFMLS1. Feature Learning2. Attention & Saliency3. Autonomous Sensory-Motor

Control4. Synthesize New WaveformsFeedback based

Beam

form

ing

TransmitRF tuning

Beam steering

Sparse EncodingFilter Low-level

Feature RepDetection & Description

System Application

System Config

Traditional “feed forward” RF system


11

Inspiration for a Better RF/digital Frontend

Low-level Feature Rep

Sparse Encoding

Abstract Feature Rep

Pattern Rec & Memory

Cognitive Task


SpeechHead turning

Eye Movement

Instinctive and overt motor control

~100 Gbps ~10 MbpsInfinite Information ~100 kbps Key characteristics for RFMLS1. Feature Learning2. Attention & Saliency3. Autonomous Sensory-Motor

Control4. Synthesize New WaveformsFeedback based

Beam

form

ing

TransmitRF tuning

Beam steering

Sparse EncodingFilter Low-level

Feature RepDetection & Description

System Application



12

Possessing the following learned properties:Receive

1. Learned features: Compact description of signal through learned features2. Attention and Saliency: discriminate important from unimportant3. Autonomous RF Control: Control configuration of RF system given task and data

Transmit4. Waveform Synthesis: Create wholly new waveform responses

RFMLS Goal is an RF System Capable ofTask-Oriented Learning from Data

Offline Training Online Evaluation

RF DataMachine Learning

Model

Task functionFeedback Model

Task function

Live streamingRF data


13

RF Machine Learning Challenges

14

RF Machine Learning ChallengesData representation & architecture Data rate/volume

Temporality Dimensionality


15

RF Machine Learning ChallengesData representation & architecture

Many signal domains & signal formats

max 𝑧𝑧 ?

Data rate/volume



16



max 𝑧𝑧 ?

Data rate/volume


224x224x3 = 150KB 1 second: 120MB (1 LTE Band)


17



max 𝑧𝑧 ?

Data rate/volume

Temporality DimensionalitySpeech

Spectrum

Human speech characteristics allow time-unrolling

conn

ecte

ddi

scon

n.

1 s 1 ms

CNN accuracy

1 s: 59%1ms: 99%



18



max 𝑧𝑧 ?

Data rate/volume



Speech

Spectrum

Human speech characteristics allow time-unrolling

conn

ecte

ddi

scon

n.

1 s 1 ms

CNN accuracy

1 s: 59%1ms: 99% DoF = 2BT

1 LTE frame = 20k

Control System Waveform Synthesis


19

RFMLS Program Structure and Technical Areas

20

RFMLS Technical Areas

Feature Learning Waveform Synthesis

Attention & Salience

Autonomous RF Sensor

Configuration

TA1 & TA2: Algorithms and Architectures

TA1: RF Forensics TA2: Spectrum Awareness

TA3: RF Hardware, System Integration, and Demonstration

Task 1A Task 1B Task 2A Task 2B


21

RFMLS Schedule

TA1 & TA2 code deliveriesTA2 (Spectrum Awareness)

Year 1(Phase 1)

Year 2(Phase 2)

Contract Award

Government testing complete

Year 3(Phase 3)

Demonstrations

TA3 (RF System Integrator)RF System delivery

TA1 (RF Forensics)

TA1 & TA2 testing completed

First GFI datasets available

TA3 Support


22

• To avoid organizational conflict of interest and to ensure objectivity in the System Integrator role, a proposer awarded under TA3 cannot be selected for any portion of TA1 of TA2, either as a prime, subcontractor, or in any other capacity from an organizational to individual level

• For the purposes of this BAA, DARPA will consider proposals from the same organization with different DUNS numbers to be separate organizations

• While proposers may submit proposals for both TA3 and TA1/TA2, the decision as to which proposal(s) to consider for award is at the discretion of the Government

Avoiding Conflict of Interest


23

TA1: RF Forensics

RF system fabrication is imperfectSolder variation, MMIC fabrication imperfections, tolerances on passive (e.g., resistors, capacitors, …) and active (e.g., amplifiers) devices

Imperfections impart intrinsic signaling characteristics on top of digital modulations

IoT Security

Software/firmware identities can be spoofed. A discriminable physical identity is needed which confirms identity.

Task 1A:Uniquely identify a wide range of devices in a large population

Task 1B:Learn a waveform modulation that allows for more effective discrimination

Feature Learning

Waveform Synthesis


24

Task 1A: RF Feature Learning

Offline training

Online evaluation

RF sample ID tag Learn RF Features

LearnClassifier

GFI Training Database(Government Furnished Information)

Samples from ~ 10,000 transmitters

RF sample ID tagRF sample ID tag

RF sample

GFI Evaluation Database

New samples from transmitters in the training database, plus a samples from new transmitters

RF sampleRF sampleStatic RF Features

Classifier

Unlimited access

Single pass

Transmitter ID

Training time limited only by RFMLS schedule

Assign a new ID if the transmitter is new

Otherwise give the ID of the transmitter


25

Task 1A Performance Scoring

Description Threshold Goal

Task 1A

PD is the probability of correctly identifying a transmitter during a future encounter

PD > 0.90 with a population of 1k transmitters


Phase 1: - No time limit on online evaluation

Phase 2: - Real-time online evaluation time limit = sample time + 1s- Additional communications protocols in GFI dataset- Real-time evaluation demonstration will be conducted using streaming data from the TA3 RF system


26

Task 1B: RF Waveform Synthesis

Objective: Learn a transmit waveform that is tailored to the unique hardware imperfections of the RF-device• Must enhance the RF fingerprint • Could be a physical-layer variant of a standard communications waveform

SDR (Software-Defined Radio)

Software DAC RF

Commanded ideal waveforms produce a baseline pre-RFMLS RF fingerprint

Idealwaveforms

New waveforms matched tohardware

imperfections


27

Task 1B Offline Learning

Task 1B system in training

Previously trained Task 1A system

Waveform under test

Feedback on discriminability from database transmitters

Waveform under test,transmitted from other SDRs

And from other SDRs

Feedback oncommunications data rate

New waveform must still be interpretable by a standard communications receiver

Same dialect transmitted by another SDR must not duplicate the RFFP


28

• If standard protocols are believed too restrictive to support RF fingerprint enhancement, proposers may propose a system that learns a new communications protocol

• More complicated (e.g., must build both transmitter and receiver)• Must match standard data rate (given bandwidth and signal-to-noise ratio)• Same waveform transmitted by another SDR must not duplicate the RFFP

Task 1B Alternative

QPSK constellation


29

Task 1B Performance Scoring


Task 1B

A transmitter increases the PD of a Task 1A system



Phase 1: Test with at least 10 identical SDRs

Phase 2: Test with at least 100 identical SDRs

How can large population performance be extrapolated from small population test data?


30

TA1 Schedule and DeliverablesPhase 1

Phase 1 Code Delivery

11 MAC* Deliver final algorithms for both training and evaluation to a government laboratory for independent evaluation

Phase 1 Final Report

12 MAC Including performance test results

Phase 2Phase 2 Code Delivery

23 MAC Deliver final algorithms for both training and evaluation to a government laboratory for independent evaluation

Real-time demonstration

24 MAC Completed demonstration of Task 1A real-time evaluation using streaming input data from the TA3 RF system. Real-time demonstration of Task 1B transmit data rate and transmitter distinguishability.

Phase 2 Final Report

24 MAC Including performance test results

* Months After Contract (or other award)Approved for Public Release, Distribution Unlimited

31

TA2: Spectrum Awareness

Task 2A:Identify all signals of a given type across 500 MHz bandwidth. Identify & characterize anomalysignals

Task 2B:Task 2A with the ability to exercise control over a hardware receiver and extend awareness over 5GHz of bandwidth

Current spectrum monitoring is on demand because building systems to reliably

monitor and classify large bandwidths is impractical

Attention & Saliency

Autonomous Sensory-Motor Control

Step 1: Find all signalsStep 2: Identify known/expected signalsStep 3: Find obvious anomaliesStep 4: Fine subtle anomalies• Low duty-cycle signals• Signals operating on top of each other• Outdoor vs. indoor• Moving vs. fixed• Incorrect data framing• …

Step 5: Characterize anomalies• Modulation• Similarity to other signals

Task 2ANo control of RF system, fixed bandwidth and direction

Task 2BControl of full RF search space

freq

spac

e

Spectrum Sharing


32

Task 2A: RF Attention and Saliency

500 MHz

Tim

e

GFI Datasets

Unlabeled Background

+

Labeled “Important Signals”

=

Development & Training Testing

Multiple types of important signals can be overlaid on any background


33

• Offline• Learn (or engineer) the capability to detect all signals• Learn to label all signals

• Online• Given the labels for important signals, find and label all important signals• Find and label anomalous signals

Task 2A: Training and Evaluation

Matched Filter Detection Labeling

Detection Encoding Importance LabelingPre-Processing

RFMLS Task 2A Receiver

Conventional Receiver


34

Task 2A Performance Scoring


Task 2A

A receiver detects, characterizes, and labels important but rare signals

500 MHz bandwidth with 10% important signals, 𝑃𝑃𝐷𝐷 > 0.5 and 𝑃𝑃

𝐹𝐹𝐹𝐹< 0.5

500 MHz bandwidth with 1% important signals, 𝑃𝑃𝐷𝐷 > 0.9 and 𝑃𝑃

𝐹𝐹𝐹𝐹< 0.5

For testing and demonstration, online evaluation must handle continuous streaming RF data


35

Task 2B

Details of funded TA3 system(s) will not be known until contract kickoff


36

• Offline• Offline learning could be accomplished with no actual RF signals or data, but only

simulated feedback (from Task 2A) on whether important signals are detected• Offline training (non-real-time or non-real-world) could result in learning that does

not apply in the online environment

• Online• With an operational Task 2A system, learn to control the TA3 system to maximize

the number of important signals found• Optimal control sequences depend on the local signal environment, and the

definition of important

Task 2B: Training and Evaluation


37

Task 2B Performance Scoring


Task 2B

A system tunes a 500 MHz receiver and detects important but rare signals. Control of other aspects of the RF system may be possible.

5 GHz tunable bandwidth with 1% important signals, PD > 0.5 and PFA < 0.5

5 GHz tunable bandwidth with 1% important signals, PD > 0.9 and PFA < 0.5


38

TA2 Schedule and Deliverables

Phase 1Phase 1 Code Delivery 11 MAC Deliver final algorithms for both training and

evaluation to a government laboratory for independent evaluation

Phase 1 Demonstration

12 MAC Completed Task 2A demonstration of offline learning and online detection and labelling of important signals. Completed Task 2B demonstration of offline learning and online control of the TA3 RF System.

Phase 1 Final Report 12 MAC Including performance test resultsPhase 2

Phase 2 Code Delivery 23 MAC Deliver final algorithms for both training and evaluation to a government laboratory for independent evaluation

Real-time demonstration

24 MAC Completed demonstration of real-time evaluation of combined Task 2A and Task 2B system using streaming input data from the TA3 RF system

Phase 2 Final Report 24 MAC Including performance test resultsPhase 3

Support Integrated Demo

24-36 MAC Support development of an integrated real-time open air demonstration with TA3

Phase 2 Final Report 36 MAC Including performance test results


39

TA1 & TA2

1 10 100 1000 10000 100000

Known-parametersIn-catalog

Known-parametersout-of-catalog

New-parametersout-of-catalog

Objects of importance

Context dependent recognition

Interactive recognition

Discriminability (population size)

Disc

rimin

atio

n Di

fficu

lty

Hand-engineered fixed-systems

Cognitive ESM

Traditional ESM

Task 1A

Task 2A

Task 2B

Task 1B


40

• Delivery of RF System 6 MAC requires use of existing components and system architectures

• Must be suitable for collecting RF data for Task 1A and Task 2A• Requires an appropriate API (Application Programming Interface) or ICD

(Interface Control Document) for Task 2B

Desired Minimum Characteristics• Tunable frequency range: 1-6 GHz

• Antenna(s) covering this frequency range• Reconfigurable instantaneous bandwidth (IBW)• Maximum IBW of at least 500MHz• Multiple (4-16) phase-coherent receivers• Resident digital beamforming

Desire additional useful degrees of freedom that can be exercised under Task 2B to enhance performance

TA3: RF System Integrator and Demonstrator


41

TA3 Schedule and Deliverables

Phase 1RF System Delivery 6 MAC Deliver 5 reconfigurable RF SystemsDataset Generation 6-12 MAC Integrate RF System to government

dataset generation infrastructure Phase 2

Integrate TA2 12-24 MAC

Support TA2 integration and real-time demonstration on TA3 hardware

Phase 3 Demo Plan 24 MAC Deliver a plan for Phase 3 demonstration of the integrated system

Phase 3Support Integrated Demo

24-36 MAC

Support development of an integrated real-time open air demonstration with TA2 performer

Final Demonstration Dry Run

33 MAC Dry run of final demonstration

Final Demonstration 35 MAC Final demonstrationPhase 3 Final Report 36 MAC


42

Program Execution

43

• Bi-weekly teleconferences & briefings• Quarterly progress review briefings / PI meeting briefings

• Kickoff meeting, final review• Proposed tasking schedule should plan for tangible results to show every quarter

• Out of cycle briefings• Technical deep dives• Transition / application interest

Additional Meetings and Deliverables


44

• The DoD does not restrict publication of fundamental research• Proposers should indicate in their proposal whether they believe the scope of

the research included in their proposal is fundamental or not• The Government will make the final decision

• Contracts for non-fundamental research will include a clause requiring DARPA review and potential edits prior to publication

• RFMLS publication is encouraged since this stimulates technology adoption and future RFML research

• Proposals may include a budget for development and presentation of academic or trade publications

Publications


45

• The Program plan does not call for down-selections or bake offs at the end of each Phase

• The proposed planning and costing by Phase (and by Task) provides DARPA with convenient times to evaluate funding options

• Factor that may affect Phase 2 & 3 funding decisions• Availability of funding• Cost of proposals selected for funding• Demonstrated performance relative to Program goals• Compatibility with other TAs and performers• Compatibility with potential DoD applications

Funding of Phases 2 and 3


46

Proposal Evaluation Criteria and Proposal Guidance


47

1. Overall scientific and technical merit2. Potential contribution and relevance to the DARPA mission3. Proposer’s capabilities and/or related experience4. Cost and schedule realism5. Plans and capability to accomplish technology transition

• The following pages contain selected tips on how to satisfy the evaluation criteria

• Refer to the BAA for complete guidance on technical and cost proposals

Proposal Evaluation Criteria


48

• The proposed technical approach is innovative, feasible, achievable, and complete.

• A specific technical solution path is proposed along with arguments why the proposed approach is expected to be successful. Analysis and trades are presented explaining why the proposed approach was selected and why alternatives were not proposed. For TA1 and TA2, a specific machine learning approach is proposed as part of an overall solution that is not dominated by expert systems. For TA3, an RF system based on specific existing hardware is proposed.

• Task descriptions and technical elements are complete and in a logical sequence leading to an endpoint supporting RFMLS goals. Proposed task elements have measureable milestones that will aid DARPA in tracking progress. Deliverables and cross-performer interfaces are clearly defined and support the RFMLS Program structure.

• The proposal identifies major technical risks and includes planned risk mitigation efforts.

1. Overall Scientific and Technical Merit


49

All TAs• Propose a specific solution path rather than a menu of potential approaches• Explain why the proposed approach was selected over alternatives• Explain why the proposed approach is feasible

TA1 & TA2• Provide quantitative arguments why the proposed approach is expected to meet

Program performance goals• Explain how the proposed approach handles:

• Complex-valued data• Features potentially across vastly different time-scales• Signal variations caused by the local environment

• Differentiate between what is pre-programmed vs. learned• Describe the proposed computational platform, and justify any proposed

equipment purchases over use of cloud computing• Provide quantitative arguments explaining why the proposed approach can handle

the expected data volumes during training and data rates for real-time evaluation

Proposal Tips – 1


50

Task 1B• Identify the standard communications protocol that you will use• Explain how your system traverses the combinatorically infinite space of

waveform design choicesTask 2B

• Describe the development and training surrogate that you will use until the TA3 system is available

• Describe how you will use the TA3 system once it is available• Describe how your solution will handle the large number of control degrees of

freedom offered by the TA3 systemTA3

• Detail the availability and maturity of the proposed RF system • Explain how each control degree of freedom might benefit Task 2B performance• Describe how your system is controlled, whether an API/ICD already exists, and

how quickly TA2 controls can modify the configuration of the system• Describe how received RF signals will be exfiltrated to adjunct TA1/TA2 processors• Outline the demonstrations planned and budgeted for Phase 3

Proposal Tips – 1.1


51

Risk Analysis and Mitigation Plan

1 21

5

2

3

3

4

4 5Probability

Cons

eque

nce

What could go wrong?A. In-house computing takes > 6 months

to execute training algorithms• Probability = 3• Consequence = 5

B. …C. …

A

Potential mitigations for Risk A:1. Port algorithms to cloud2. Develop second algorithmic solution

selected for computational efficiency

A

A

Proposal should include risk mitigation tasks, budget, and schedule so that the resulting plan has no risk remaining in the red zone


52

The potential contributions of the proposed effort are relevant to the national technology base. Specifically, DARPA’s mission is to make pivotal early technology investments that create or prevent strategic surprise for U.S. National Security.

Proposal Tip: Consider the perspective of the proposal reviewer that must describe how your proposal contributes to the DARPA mission

2. Potential Contribution and Relevance to the DARPA Mission


53

Similar efforts completed/ongoing by the proposer in this area are fully described, including identification of other Government sponsors. The proposed team has the expertise to manage the cost and schedule. The proposer's prior experience in similar efforts demonstrates an ability to deliver products that meet the proposed technical performance within the proposed budget and schedule. TA1 and TA2 proposers have prior experience developing and running ML algorithms on large data volumes. TA3 proposers have prior experience integrating and field testing complete RF systems.

3. Proposer’s Capabilities and/or Related Experience


54

TA1 & TA2• TA1 performers do not require (but may benefit from) prior RF experience• Machine learning experience is of equal or higher importance than RF systems

experience• Experience developing machine learning algorithms and architectures is far more

valuable than experience using commercial machine learning platforms• The proposal should describe relevant experience and cite evidence such as

publications• Experience with RF systems and data processing is also relevant and should be

describedTA3

• TA3 proposers should describe their prior experience developing, integrating, and testing complex RF systems

• Proposals should emphasize complete end-to-end functional systems rather than sub-systems (e.g. receivers) and components (e.g. MMICs)

• The description of this prior experience should give DARPA confidence that the proposer has the capabilities to integrate TA2 for real-time functionality, and to demonstrate system capabilities through outdoor testing

Proposal Tips – 3


55

• Funding the proposed effort would not consume a disproportionate fraction of the RFMLS Program budget.

• DARPA recognizes that undue emphasis on cost may motivate proposers to offer solutions that are unlikely to fulfill RFMLS goals, or to staff the effort with junior personnel. DARPA discourages such cost strategies.

• The proposed staffing and schedule is consistent with the proposed tasking and technical milestones.

• The proposed costs are realistic for the technical and management approach and accurately reflect the technical goals and objectives of the solicitation. The proposed costs are consistent with the proposer's Statement of Work and reflect a sufficient understanding of the costs and level of effort needed to successfully accomplish the proposed technical approach. The costs for the prime proposer and proposed subawardees are substantiated by the details provided in the proposal.

• The proposal identifies major cost and schedule risks and includes planned risk mitigation efforts.

4. Cost Realism

Abridged text – See BAA for full text of evaluation criteria 4Approved for Public Release, Distribution Unlimited

56

All TAs• Phases 1, 2, and 3 must be separately planned and priced• Include a detailed breakdown of technical tasking, with cost and schedule for

each sub-taskTA1 & TA2

• Your proposal must include both Task A and Task B• Task A and Task B must be separately planned and priced• Include a detailed schedule showing the interactions between Task A and Task B

Task 1B• Identify and budget for an SDR, and if it is not commercially available then

budget for delivery of an SDR for Government testingTask 2B

• Describe your plan for interacting with TA3 personnelTA3

• If API/ICD development is required, include a schedule of development• Describe your plan for interacting with TA2 staff

Proposal Tips – 4


57

The proposed intellectual property (IP) rights structure does not limit the Government’s ability to transition the technology to Government applications. The proposal demonstrates capability to transition the technology to the research, industrial, and/or operational military communities in such a way as to enhance U.S. defense.

Proposal Tip: Proposals which include IP rights which limit the government’s ability to transition the technology may reduce the overall evaluation of the proposal

5. Plans and Capability to Accomplish Technology Transition


58

Contracting

59

RFMLS Industry Day Agenda

9:00-9:15 Agenda and Logistics9:15-9:35 Vision9:35-9:55 RF Machine Learning Challenges9:55-10:50 Program Structure and Technical Areas10:50-10:55 Program Execution10:55-11:15 Break11:15-11:40 Proposal Evaluation Criteria and Proposal Guidance11:40-12:00 Contracting12:00-1:00 Lunch Break1:00-4:00 PM Meetings / Teaming Time4:00-5:00 Question & Answer Session

August 31, 2017


www.darpa.mil

60

Date post:	30-Mar-2018
Category:	Documents
Upload:	dangtram
View:	260 times
Download:	15 times

RF Machine Learning Systems (RFMLS) - DARPA · PDF fileRF Machine Learning Systems (RFMLS) ......

Documents