Progress report

2012/05/01 匡建慈

Reminding

פ Goals : to build a multi-core platform with Hadoop environment

Base board & FPGA

& fiber ports

Tegra2 Application layer

Transaction layer

Physical Layer

Linux Device Driver

DMA in Tegra2Router circuit in FPGAFiber link

User programUser API

What to do and what we have

פ What we have פ Tegra2 software program co-work with FPGA to do simple work .פ A circuit example with fiber communication

פ What to do פ a example by combining examples we have to achieve a

framework from software , hardware , and fibers . ( O )פ Example interface unified פ Achieve API package פ Interrupt mechanism

פ Porting Hadoop environment on proposed framework . ( X )

פ Combined example block diagram

DMA

TX_Buf

RX_Buf

TX_IF

RX_IF

GTPBASE

Fiber link

Linux Devic

e Driver

User Prog.

Circuit layer SoftwareDriver

DMAIF

Clk_wiz

clk_120mBASE_CLK

פ API packageפ Global variable : device number , buffer pointerפ init : open device , memory mappingפ Finalize : close device , memory um-mapping .פ FPGAWrite : write buf_w into circuit layer by DMAפ FPGARead : Read data from circuit by DMA and store into buf_r

פ header packet mechanism פ In future , it wiil be applied on

APIs

פ Interrupt mechanismפ Circuit layer :

פ pull up MEM_R/W_Ready to setup an interrupt to CPUפ A interrupt means it’s ready for the next user read/write call

פ Driver layer פ Driver code in original release might be got stuck due to mutex

usage .פ According to previous work on MultiARM , rewrite driver by a

kthread interrupt handler and two wait_events

25 ns

Result

2-node test

פ A single SendRecv program running on 2 nodesפ Add a ‘!’ until the length is larger than specified value

Receiver

Sender

Next

פ Try to setup Hadoop environment on proposed framework .פ Do on my laptopפ Setup JAVA on MaCube פ Setup Hadoop environment on MaCubeפ Modify the calling interface to my APIs

Progress report

2012/05/15 boy13

Last week

פ Completedפ Setup programming model and achieve framework covered with

software , hardware , GTPs .פ Combining examples פ API package פ Interrupt mechanism

פ To doפ Setup Hadoop on proposed framework

פ Setting Hadoop on this laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Modify calling interface by my APIs X

Setting Hadoop on my laptop

פ Single node settingפ Download Hadoop and jdk and extract them .פ <Hadoop Path>/conf/hadoop_env.sh

פ Export JAVA_HOME=<JdkPath>

פ <Hadoop Path>/conf/core-site.xmlפ hadoop.tmp.dir <a path to put /tmp>פ fs.defailt.name hdfs://localhost:9000פ Dfs.replication 1

פ <Hadoop Path>/conf/mapred-site.xmlפ Mapred.job.tracker localhost:9001

פ More on http://changyy.pixnet.net/blog/post/25245658

פ Hadoop example executionפ HDFS format

פ <Hadoop Path>/bin/hadoop namenode -format

פ Start and stopפ <Hadoop Path>/bin/start-all.shפ <Hadoop Path>/bin/stop-all.sh

פ HDFS operation פ <Hadoop Path>/bin/hadoop dfs –mkdir inputפ <Hadoop Path>/bin/hadoop dfs –lsפ <Hadoop Path>/bin/hadoop dfs –put <filepath> input

פ Hadoop example wordcount executionפ <Hadoop Path>/bin/hadoop jar hadoop-0.20.1-examples.jar

wordcount input output

פ See execution resultפ <Hadoop Path>/bin/hadoop dfs –cat output/*

פ Multi-node settingפ Hosts

פ /etc/hosts : A table record machine name and its IPפ /etc/hostname : machine name

פ ssh connections פ sudo apt-get install openssh-server

פ <Hadoop Path>/confפ slaves : machine name of data-nodesפ masters : machine name of name-node

פ start-all.sh

פ After start-all.shפ <Jdk Path>/bin/jpsפ <Hadoop Path>/bin/hadoop dfsadmin –report

פ More on : http://changyy.pixnet.net/blog/post/25612440

Setting JAVA Runtime Environment

פ cp /media/sda1/ejre1.6.0_32.tar.gz ./פ tar -xjf ejre1.6.0_32.tar.gzפ export PATH=/home/root/Desktop/ejre1.6.0_32/bin:$PATH

Next to do

פ Try to setup Hadoop environment on proposed framework .פ Do on my laptopפ Setup JAVA on MaCube פ Setup Hadoop environment on MaCubeפ Modify the calling interface to my APIs

Progress report

2010/05/29 boy13

Recently

פ Goals : to build a multi-core system with hadoop environment

פ Work items פ Setup programming model and achieve framework covered with

software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism

פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube

Router mechanism

פ A Light-Weight Parallel Router for FPGA-based Networks-on-Chip

פ Department of ECECS , University of Cincinnati, 2005

Application layer

Transaction layer

Physical Layer

Linux Device Driver

DMA in Tegra2Router circuit in FPGAFiber link

User programUser API

Router mechanism

פ 2-D mesh architecture

פ Routing mechanism פ XY routing

פ Portפ Input Channel

פ FIFO interfaceפ Be requested from Output Channel of a port in another routerפ Request to X-bar switch to access Output Channel

פ Output Channel

פ Crossbar Switch

פ For out architecture פ 2-D mesh => 3-D meshפ Port => Local Port , Fiber Port

פ Local Port : interface for FPGA and DMA in tegra2 פ Fiber Port : interface for fiber link in proposed framework

Base board & FPGA

& fiber ports

Tegra2

Setting hadoop in proposed frameworkפ Setting up namenode

פ applet not found

פ What is applet ?פ JRE v.s. Java Applet

פ By several weeks asking ,I conclude that JAVA Applet isn’t supported in embedded JAVA .

פ Setting up datanode in MaCubeפ Laptop plays namenode role in this work .

פ Setting ssh connection פ Pre-requite : openssl , zlib

פ Download source package פ Extract it to local directoryפ ./configure –helpפ ./configure <options for cross compile>פ Modify makefile manually if necessary פ make && make install

פ Error occur when I key make install for opensshפ Source pack : 5.7 , 5.9 פ Toolchain : arm-2009q3 , cc.2011.03

Next

פ Router mechanismפ Complete design for Local port , Crossbar switch פ Change local port with fiber port פ Complete 2x2 2-D mesh architecture

פ Setup hadoop environment פ Build openssh and setup connectionפ Try datanode start service

Progress report

2012/06/19 boy13

Recently

פ Goals : to build a multi-core system without hadoop environment

פ Work items פ Setup programming model and achieve framework covered with

software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system

פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service

Router mechanism

פ Router פ Input Channel , Output Channelפ Input Request mechanism , Round-Robin Arbitratorפ Parallel Router by cross-bar switch

פ Testing pattern פ Local port Oפ Local port + Router Oפ Local port + Router + 1 fiber port Oפ Local port + Router + 4 fiber port O

DMA

Local port

Router

East port West port

GTP_J4

North portSouth port

GTP_J5

id

Multi-core system

פ So far , only 1-1 send-recv pairs , and no packet overlapping occur .

פ Read mechanism in driver should be designed פ When read interrupt comes

פ configure DMA and put data into a bufferפ Wakeup kthread (interrupt handler)

פ Kthreadפ Put data in buffer in a list kept in driver

פ Read callפ Search list in driver פ If found , copy data into read buffer API allocatedפ Otherwise , sleep

פ Re-check circuit designפ Whether data not sent but be replaced or notפ Buffer empty register

Setting hadoop environment

פ Hadoop settingפ Download hadoop source and JRE into MaCubeפ Modify <hadoop path>/conf/ * .xmlפ Build ssh connectionפ ./start-all.shפ ./bin/hadoop <execution commands>

פ 2 use-caseפ MaCube play both namenode and datanode rolesפ MaCube plays datanode role and laptop/PC act as namenode

פ Problem encountered for both datanode and namenode roleפ bash : no such file or directory

פ What is bash ?פ ln –s /bin/sh /bin/bash

פ bash : applet not found

פ By several weeks googling and asking , I conclude that JAVA Applet isn’t supported in embedded JAVA .

פ Taiwan Hadoop Forumפ http://hadoop.nchc.org.tw/phpbb/index.php?sid=021ae10a0e0609

ab818b3e7062aaa314

פ Embedded JAVA , no bash => no applet => hadoop can’t be built .

פ So ……

Namenode software service

פ Motivation : hadoop is not friendly with embedded system .

פ We build a simple software service of cloud systemפ A simple service to submit job

פ Package source code , data , and execution argumentsפ cross-compile source codeפ Send package to datanode scheduled פ Result returned

פ Graphics user interface and it’s easy to use.פ Embedded JAVA !!פ New thread and system call to execute program received

פ Simple distributed file systemפ Connect with student of Prof. Shiao

To do

פ achieve multi-core system hardware frameworkפ Complete read mechanismפ Put fiber port from 4 to 6

פ Design namenode software service פ A quick job submit flowפ Easy to use and graphical

Progress report

2012/07/10 boy13

Recently

פ Goals : to build a multi-core system

פ Work items פ Setup programming model and achieve framework covered with

software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system

פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service

פ Demo items

פ Multi-core systemפ Transportation testing

פ X-axis transportation פ Y-axis transportation פ 2-D mesh transportation

0 1

2 3

0 1 2 3

פ Multi-core systemפ Serial transportation

פ Motivation : sometimes a byte data shift

פ Multi-core systemפ Serial transportation problem

פ Reference clock skewing

פ Solutions :פ Sink : make the initiator and target correct the timing when initial

the transactionפ Re-send : if data check is error by some mechanism , then request

initiator to resend

פ The author of GTP_EXAMPLE says , it’s locked when the first sink , and can’t be unlocked .

פ => only can try re-send method .

פ Method thinking פ do something when error has checked at the first time .

פ Ack/Nak mechanism . פ Deadlock possible ?

TX_IF

RX_IF

GTPCORE

TX_IF

RX_IF

GTPCORE

פ Namenode software serviceפ TCP communication

פ Try : a simple chat room program

פ Shell program

Server

Initial socket

Listen client

Create thread to message passing

Create receive thread

Message recv

Take mutex and queue it

Check queue is empty

Take mutex , pop queue ,Socket send

Client

Initial socketGet

connection

Create thread to recv

chatSocket recv

Print

פ Demo items פ Demo item 1 :

פ Using Ethernet to do Map-Reduce or SPCP

פ Demo item 2 (possible) : פ Using Ethernet to transport program and data פ Using fiber to communication with components

1 2 3

4 5 6

7 8

Progress report

2012/08/01

Recently

פ Goals : to build a multi-core system

פ Work items פ Setup programming model and achieve framework covered with

software , hardware , GTPs .פ Combining examples Oפ API package Oפ Interrupt mechanism Oפ Router mechanism Oפ Multi-core system

פ Setup Hadoop on proposed frameworkפ Setup up on my laptop Oפ Setting JAVA Runtime Environment on MaCube Oפ Setting Hadoop environment on MaCube Xפ Namenode software service

פ Demo items

Recently

פ Transport testing

פ Middle-ware implementation

פ Demo items implementation

Transport testing

פ 2-D mesh architecture

פ X_testפ 0,1 OKפ 4,5 X -> OK

פ Y_testפ 0,4 X -> OKפ 1,5 X -> OKפ Synchronous problem occurred frequently .

0,0(0)

1,0(1)

1,0(4)

1,1(5)

Middle-ware implementation

פ Shell program(JAVA) , TCP Agent(C++) , TCP Core(C++)פ connection built OK , but no data received

פ JAVA object serialization

פ It’s OK to ping Macube from Vmware , but it doesn’t work to ping Vmware from Macube

פ Double-system laptop

פ Viewing SPCP sources

פ => porting SPCP onto Macubeפ Cross-compile core program and takes related filesפ Possible problem : speed gap between PC and Macube

Demo item implementation

פ Item 1 : porting SPCP

פ Item 2 : simple puzzle (possible)פ Ethernet version

פ Packet size , header configurationפ no puzzle change , 2 cores on PC , 4 cores on Macubesפ Speed gap

1 2 3

4 5

JAVA UI

Upload

System call and wait

Server

Core

Core

Core

Core

TCP/IP connection

פ Item 2 : simple puzzle (possible)פ Fiber version

פ Serial transport synchronous problemפ Re-send , Sync at transport initial

פ re-send mechanism

1 2 3

4 5

TX_sendRX get Ack

timeout

done

Re-send

RX_recvCRC_error =

1 discard

Send ack packet

TX_sendAck

RX_getAck

CRC_error = 1

discard

Thanks for your attention !